[Twisted-Python] Simultaneous Blocking Operations and Twisted

[Jean-Paul Calderone]

On Fri, 30 Mar 2007 12:01:27 +0100, Matthew Glubb <matt at zgroupplc.com> wrote:
>Hi All,
>
>I am new to twisted and python. I am evaluating the possibility of  using it 
>for an application that I am writing. The framework looks  very 
>comprehensive (I am also enjoying the python language :) but I  have some 
>questions regarding deferred objects and the twisted web  server itself.
>
>My application essentially comprises the following:
>
>1. HTTP request is received by the server.
>2. Server creates a listening socket on an arbitrary port and waits  for n 
>seconds for incoming data
>3. Data (or timeout message) is returned to the client by way of the  HTTP 
>response.
>
> [snip]
>
>I was wondering whether it is feasible to use the twisted framework  for 
>this application. I have been playing with the framework and I  can't see a 
>way to service what are essentially thousands of requests  that perform 
>blocking operations simultaneously. I have noted the  twisted documentation 
>on threads but I was hoping, for the sake of my  sanity, that I might be 
>able to manage this using an asynchronous  approach. Essentially, what I 
>think I am looking for is a waitFor()  on a deferred callback.

There's no way to block until an event occurs.  Blocking would prevent the
reactor from noticing the event or informing you about it. :)

Fortunately you don't need to resort to threads, either.

The conventional way to do this would be to write a function which sets up
the listening port and returns a Deferred which will eventually be called
back with the result (presumably the data which some client will eventually
send to it).  Then, using reactor.callLater, set up a timed call which will
tear down the listening port after the timeout has expired and errback the
Deferred to indicate no data arrived within the timeout period.

Actually, that's the only way to do it. :P  The conventional implementation
would be to do this using explicit callback functions in the style you may
have noticed in many Twisted examples.  There are other ways which allow you
to write code in a different style which some people prefer, but it's just
a difference in spelling, the same thing is happening underneath.

>
>As an aside, is it possible to compile platform specific python 
>applications as small binaries? Or does the interpreter (is it  required?) 
>mean the memory consumption is prohibitive?

Generally the only thing that's possible is to wrap up the whole application
with the whole interpreter and some subset of the standard library in an
executable.  This doesn't sound much like what you're after, though.

Jean-Paul