[Twisted-Python] Setting a timeout for HTTPDownloader

[Bob Ippolito]

On Tuesday, Aug 12, 2003, at 09:48 America/New_York, Glyph Lefkowitz 
wrote:

>
> On Tuesday, August 12, 2003, at 06:27 AM, Edd Dumbill wrote:
>
>> I'd be grateful for pointers on how to implement this, with reference 
>> to
>> twisted/web/spider.py.  Essentially what I want is a timeout to cause 
>> an
>> error that will result in SpiderSender.downloadFailed being invoked,
>> pretty much as for any other error relating to the HTTP connection 
>> does.
>
> I certainly haven't tested this, but in your SpiderSender subclass, 
> you probably want something like
>
> 	myTimeout = 120 # seconds
> 	def notifyDownloadStart(self, uri):
> 	    reactor.callLater(self.myTimeout, self._timeOut, uri)
> 	    # rest of notifyDownloadStart
>
> 	def _timeOut(self, uri):
> 	    if self.downloaders.has_key(uri):
> 	        self.downloaders[uri].loseConnection()

So how do you make this work based upon a *request* to make a download 
stop, rather than a predetermined amount of time?  ;)

-bob