[Twisted-Python] Setting a timeout for HTTPDownloader

[Glyph Lefkowitz]

On Tuesday, August 12, 2003, at 06:27 AM, Edd Dumbill wrote:

> I'd be grateful for pointers on how to implement this, with reference 
> to
> twisted/web/spider.py.  Essentially what I want is a timeout to cause 
> an
> error that will result in SpiderSender.downloadFailed being invoked,
> pretty much as for any other error relating to the HTTP connection 
> does.

I certainly haven't tested this, but in your SpiderSender subclass, you 
probably want something like

	myTimeout = 120 # seconds
	def notifyDownloadStart(self, uri):
	    reactor.callLater(self.myTimeout, self._timeOut, uri)
	    # rest of notifyDownloadStart

	def _timeOut(self, uri):
	    if self.downloaders.has_key(uri):
	        self.downloaders[uri].loseConnection()