[Twisted-Python] which async framework?

[Laurens Van Houtven]

Hi Chris,


It will probably not surprise you that I like Twisted :-)

On Wed, Mar 12, 2014 at 8:28 AM, Chris Withers <chris at simplistix.co.uk>wrote:

> So, I see python now has a plethora of async frameworks and I need to try
> and pick one to use from:
>
> - asyncio/tulip
> - tornado
> - twisted
>

Twisted has a thing called Cyclone, which I hear (but that's only hearsay)
gives you Tornado's API on top of twisted, so you get all of the stuff
below for free.


> From my side, I'm looking to experimentally build a network testing tool
> that will need to speak a fair few protocols, both classic tcp and
> multicast-based, and have a web api living on top of it that most likely
> will have a websocket for pumping data to the browser. It'll also need to
> write out JUnit-compatible xml results, but that's like the easiest bit ;-)
>

I don't know which protocols you are interested in specifically; later on
in the e-mail you mention that it's *probably* not already extant. Twisted
comes with a huge number of protocols already implemented. It has TCP
support (duh) as well as an interface for UDP multicast.

Twisted has a third party project called txsockjs which works excellently,
both by itself and in the context of other (HTTP) resources. sockjs is a
protocol which is basically "websockets, damnit", even when the consumer is
a bad browser like IE6.


> I'd like to be able to serve the rest of the web api using a pyramid wsgi
> app if possible, and I'd like to be able to write the things that process
> requests in and validation out in a synchronous fashion, most likely
> spinning off a thread for each one.
>

Twisted comes with a threadpool-backed WSGI server. (When I say
threadpool-backed I mean that the WSGI requests are handled in threads; the
IO itself of course comes from the Twisted reactor).


> The protocols are all financial (do we really not have a pure-python FIX
> library?!) but none are likely to have existing python implementations.
>

I find twisted to be a great tool for writing protocol implementations. I
have written tools for querying all sorts of gnarly proprietary protocols
over all sorts of gnarly transports (packet radio; it's totally a thing),
and more recently for doing crazy things like multiplexing stream
transports over existing RPC protocols. (Like, you see a local port come
up, and that actually creates a virtual stream connection over an existing
RPC thing to some virtual server on the other end of the wire:
https://www.youtube.com/watch?v=W_jEIvugwes).

Twisted has had so many people write so many protocols in it that also the
testing tools (MemoryReactor, StringTransport) are great. Especially if you
are writing something very close to a wire protocol you will undoubtedly
enjoy those amenities. There's also tons of composable things for receiving
delimited lines, nestrings, etc. It's hard to tell what you will be looking
for since I don't know details about your protocol, but having written more
than a few protocol implementations I'm going to wager a guess and say
Twisted has it or a third party thing for twisted has it.

Even though I mostly write Clojure now, I still write my networking stuff
in Twisted. Macros are cool. Eleven years worth of battle-tested code is
better.

That said, tulip is nice in that you can also write protocol
implementations that will look similar :-)


> How should I pick between the options? What would people recommend and why?
>

Twisted, emphatically and without reservation, for all the above reasons.
It's stable. All the stuff you need has been tried and tested extensively
in production. It runs on PyPy, and usually a damn sight faster than on
CPython, too.

A common criticism of Twisted is that it "takes over" your codebase. I am
speaking at PyCon in about a month to demonstrate that that isn't true. (It
just looks that way, because once people use it, they don't want to go
back... ;-))

hth
lvh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20140312/309d5ae9/attachment-0002.html>