[Twisted-web] NEWBIE: how to use only the networking core of twisted-web

[Andrew Bennetts]

On Fri, Jan 02, 2004 at 03:09:21PM -0800, dave giffin wrote:

[snip: Currently using Medusa, but having problems]
> 
> I'm looking for a web server that uses threads (which
> I think twisted-web does), but I want to use just the
> HTTP server core of it, not the part that handles the
> file locating, opening and the executing of python
> scripts (the part that deals with .rpy scripts which
> have to import certain classes). 

Twisted Web doesn't need threads for concurrency -- but there's no reason
why code using Twisted Web couldn't use them.  But then, this also applies
to Medusa -- that's how Zope works, for instance :)

> I just want to move my software from medusa to
> twisted-web, so I need to know which function is
> called when a request is recieved.

Two options spring to mind:
    - Create a simple twisted.web.server.Site with a single Resource with
      isLeaf = 1
    - Avoid twisted.web entirely, and use twisted.protocols.http.  

Oh, and a third:
    - Make your code no need threads anymore.  This is probably easier to do
      with Twisted than you expect.

If you want to stick with threads, I'd recommend figuring out how to do it
with Medusa.  Twisted is a fine framework, but if you want to basically
ignore its features (by blindly using threads for concurrency and by
bypassing the bulk of Twisted Web) then it's probably going to frustrate you
more than help, because it's not really designed for that use case.

That said, here's where to start looking for how to do it:

With method one (Twisted Web with a simple Resource), something like this:

    # Completely untested!!
    from twisted.web.resource import Resource
    from twisted.web.server import NOT_DONE_YET
    from twisted.web.error import ErrorPage
    from twisted.internet.threads import deferToThread
    from twisted.protocols import http
    from twisted.python import log
    from exMedusaCode import process_request
    
    class ThreadDispatchResource(Resource):
        isLeaf = 1
        
        def render(self, request):
            d = deferToThread(process_request, request)
            d.addCallback(self._cb, request)
            d.addErrback(self._eb, request)
            return NOT_DONE_YET

        def _cb(self, result, request):
            request.write(result)
            request.finish()

        def _eb(self, failure, request):
            log.err(failure)
            body = ErrorPage(http.INTERNAL_SERVER_ERROR,
                             "See log file for details").render(request)
            request.write(body)
            request.finish()

You'll probably want to munge the request object from the standard Twisted
one into whatever your Medusa code (which I'm assuming is called
'process_request') is expecting.

With method two, you'd probably want to make your own
twisted.protocols.http.Request subclass, which means overriding
HTTPFactory's buildProtocol to set the HTTPChannel's requestFactory to your
new subclass (see twisted.web.server.Site for an example).  The actual
mechanism will be pretty similar -- use deferToThread, and add a callback to
call request.finish() when code in the thread is done.

> This way, I should be able to switch easily.

If you just want a simple thread-dispatcher for your Medusa code, then
Twisted isn't really what you want.  I'd recommend looking at how ZServer in
Zope extends Medusa to do it.

-Andrew.