[Twisted-Python] sending large files from web2 http server

[Markus Wanner]

Hi,

I'm trying to stream longish data via web2, but experience sudden stalls
in data transfer, followed by a connection abort after a certain
timeout. I can't completely reproduce the issue, yet, but figured that
the size of the blocks I'm returning via the stream's read method
affects the failure rate (an stream.IByteStream implementation).

I've tried with chunk sizes between 1 MiB down to 256 bytes. At that
rate, I suddenly get the following unhandled errors, which don't seem to
have to do much with my code.

2009/03/05 16:02 CET [-] Unhandled error in Deferred:
2009/03/05 16:02 CET [-] Traceback (most recent call last):
	  File "/usr/lib/python2.4/site-packages/twisted/internet/defer.py",
line 267, in unpause
	    self._runCallbacks()
	  File "/usr/lib/python2.4/site-packages/twisted/internet/defer.py",
line 307, in _runCallbacks
	    self.result = callback(self.result, *args, **kw)
	  File "/usr/lib/python2.4/site-packages/twisted/internet/defer.py",
line 271, in _continue
	    self.unpause()
	  File "/usr/lib/python2.4/site-packages/twisted/internet/defer.py",
line 267, in unpause
	    self._runCallbacks()
	--- <exception caught here> ---
	  File "/usr/lib/python2.4/site-packages/twisted/internet/defer.py",
line 307, in _runCallbacks
	    self.result = callback(self.result, *args, **kw)
	  File "/usr/lib/python2.4/site-packages/twisted/internet/defer.py",
line 271, in _continue
	    self.unpause()
	  File "/usr/lib/python2.4/site-packages/twisted/internet/defer.py",
line 267, in unpause
	    self._runCallbacks()
	  File "/usr/lib/python2.4/site-packages/twisted/internet/defer.py",
line 321, in _runCallbacks
	    self.result = failure.Failure()
	  File "/usr/lib/python2.4/site-packages/twisted/python/failure.py",
line 205, in __init__
	    parentCs = reflect.allYourBase(self.type)
	  File "/usr/lib/python2.4/site-packages/twisted/python/reflect.py",
line 480, in allYourBase
	    accumulateBases(classObj, l, baseClass)
	exceptions.RuntimeError: maximum recursion depth exceeded


Are there any known problems with web2 and large files (not that 64 MiB
is *that* large...). Any hint regarding the above error? Is the same to
be expected with a chunk size of 8 KiB and a file of 2 GiB size (as that
would result in the same amount of chunks)?

Any hints would be greatly appreciated.

Kind Regards

Markus Wanner