[Twisted-Python] Twisted protocol as Django data source

[Esteve Fernandez]

On Wednesday 27 May 2009 21:48:25 Alex Clemesha wrote:
> I wrote up some thoughts on this here:
> http://clemesha.org/blog/2009/apr/23/Django-on-Twisted-using-latest-twisted
>-web-wsgi/
>
> basically it comes down to running Django off the very latest WSGI
> code (in the trunk still)
> found in twisted.web, which I've found to work very well.
>
> Also see here:
> http://blog.dreid.org/2009/03/twisted-django-it-wont-burn-down-your.html

Having Django running on top of Twisted.web doesn't make Django a normal 
Twisted application. You'll have to use reactor.callFromThread in a lot of 
places to maintain thread-safety, since most of the Twisted code isn't.

Like JP has pointed out, a solution based around RPC is a viable option. It's 
pretty straightforward, safe and performant.

My take (which uses Thrift for RPC, could have imagined that? :-)):

- define a Thrift spec file that describes a service with the methods that 
your Django application will issue (like queryResults(), readData(), etc.)

- generate code using the Thrift compiler for both blocking and non-blocking 
Python:
$ thrift --gen py (for regular Python)
$ thrift --gen py:twisted (for Twisted)

beware that both commands will dump code to a directory called gen-py, which 
gets overwritten each time

- create a normal Twisted application which speaks your protocol on one end 
and Thrift on the other. Use whatever transport you like, if you want to make 
it ultra fast use unix sockets and a tmpfs filesystem (e.g. /tmp in most 
Linux distributions)

- connect from your Django application to the Twisted Thrift service and issue 
whichever calls you defined in the first step

I have some code sitting around that does this (for txAMQP, in my case), I'll 
try to upload it somewhere.

Cheers.