[Twisted-web] Re: [Twisted-Python] WSGI Thread-management strategy
Jim Fulton
jim at zope.com
Fri Dec 16 10:27:15 MST 2005
James Y Knight wrote:
> (BTW, the correct mailing list for twisted webbish stuff is twisted-
> web at twistedmatrix.com)
> On Dec 15, 2005, at 4:22 PM, Jim Fulton wrote:
>
>> The strategy used by twisted WSGI, as I understand it, doesn't meet
>> our needs. Currently, a thread is created for each request. The total
>> number of threads is throttled, I gather using a general Twisted
>> thread limit. WSGI applications are called as soon as input headers
>> have been received completely. An application may be called before all
>> body input is received. We need application calls to be delayed until
>> all request input has been received,
>>
>> [...]
>>
>> I propose that the default thread-management strategy should be to delay
>> calling an application until all request input has been received. If
>> this isn't the default, then there should at least be an option to get
>> this behavior. (Of course, the buffering strategy needs to be clever
>> enough to switch to a file when the input gets over some size.)
>
>
> Sounds sensible,
Does this mean you will make this the default in the future?
> and is doable external to the WSGI wrapper. Here's a
> little bit I whipped up. (works on the 2.1.x branch and head). Could be
> smarter, by starting out the buffer in memory and switching to a file
> if necessary. Also shows off a couple of minor bugs I need to fix. :)
>
> def simple_wsgi_app(environ, start_response):
> print "Starting wsgi app"
> start_response("200 OK", [('Content-type','text/html;
> charset=ISO-8859-1')])
> data = environ['wsgi.input'].read()
> return ['<pre>', data, '</pre>']
>
>
> class Prebuffer(resource.WrapperResource):
> def hook(self, ctx):
> req = iweb.IRequest(ctx)
> temp = tempfile.TemporaryFile()
> def done(_):
> temp.seek(0)
> # Replace the request's stream object with the tempfile
> req.stream = stream.FileStream(temp)
> # Hm, this shouldn't be required:
> req.stream.doStartReading = None
> return stream.readStream(req.stream, temp.write).addCallback (done)
>
> # Oops, fix missing () in lambda in WrapperResource
> def locateChild(self, ctx, segments):
> x = self.hook(ctx)
> if x is not None:
> return x.addCallback(lambda data: (self.res, segments))
> return self.res, segments
>
> if __name__ == '__builtin__':
> from twisted.application import service, strports
> from twisted.web2 import server, channel
>
> res = Prebuffer(wsgi.WSGIResource(simple_wsgi_app))
>
> site = server.Site(res)
> application = service.Application("demo")
>
> s = strports.service('tcp:8080', channel.HTTPFactory(site))
> s.setServiceParent(application)
This is greek to me. I'd forward this to the people who did the
Twisted Zope integration.
Thanks!
Jim
--
Jim Fulton mailto:jim at zope.com Python Powered!
CTO (540) 361-1714 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org
More information about the Twisted-web
mailing list