[Twisted-Python] Setting a timeout for HTTPDownloader

[Edd Dumbill]

On Tue, 2003-08-12 at 14:48, Glyph Lefkowitz wrote:
> I certainly haven't tested this, but in your SpiderSender subclass, you 
> probably want something like
> 
> 	myTimeout = 120 # seconds
> 	def notifyDownloadStart(self, uri):
> 	    reactor.callLater(self.myTimeout, self._timeOut, uri)
> 	    # rest of notifyDownloadStart
> 
> 	def _timeOut(self, uri):
> 	    if self.downloaders.has_key(uri):
> 	        self.downloaders[uri].loseConnection()
> 

Thanks, I'll try that.  I think it was the loseConnection() bit I was
missing.

My interim solution is a bit nuttier, and I'm not actually convinced it
works, either, but it did allow me to find a bug with 302 redirect loops
(issue 99!)

class TimeOutHTTPPageDownloader(client.HTTPPageDownloader,TimeoutMixin):
    def connectionMade (self):
        self.setTimeout (30)
        client.HTTPPageDownloader.connectionMade (self)

class HTTPDownloader(client.HTTPDownloader):

    protocol = TimeOutHTTPPageDownloader
   

...

-- Edd