[Twisted-Python] Twisted Advice

[Paul Wilson]

Hi!

I am researching technologies for building an application development
framework with Python, and have been interested in Twisted to manage
the network communication side of things. However, I read something
yesterday that appears to undermine the applicability of twisted for
my work. I've done quite a bit of reading around the subject since,
and have ended up confusing myself!

A remote client accesses my server to get instructions on how to
progress a customer "interaction." The remote client is itself a
server driven by a telephone call and keytone input. The remote client
could be servicing N calls (channels) simultaneously, and thus my
server needs to support N simultaneous TCP connections. Pretty
standard stuff.

I would like to provide an environment where developers can write call
servicing applications with no knowledge of the network. I had rested
on a coroutine approach, whereby a developer could write something
like this:

   response, events = (yield getKeyPress(SomePromptAnnouncement))

With the response, the developer can do an undefined number of things,
taking an undefined amount of time. During this time, management
events can arrive on a separate port, to which the developer gets
knowledge of through the yield statement.

On the other side of this generator is a scheduler which takes
instructions such as "getKeyPress" and passes it through twisted to
the remote client, such that it can play the announcement and get a
keypress. The scheduler gets the response and sends to back to the
generator, along with any events that have arrived on this separate
port.

When a call/connection arrives, the client will send an opening
line(s) to which I was specify the correct LineRecieved method. This
will trigger some scheduler code defined elsewhere via a deferred,
which will prompt the developer's code for some instructions, such as
"PlayPrompt". Am I correct in thinking that while a developer's code
is executing, all other connections are paused, and that the twisted
server will not accept new connections until it returns?

My original assumption was that Twisted would spawn a new thread
within which the scheduler would be set to run to manage the
communication for the duration of the customer call/interaction.
However, the approach taken by twisted is that if the developer's code
got itself in a muddle or infinite loop or took a very long time
accessing a database, the rest of the application would just be
frozen.

I wrote a quick test app that defers to a function that sleeps then
returns its response line. During the 20 seconds that the test server
took to return a line, the server would not accept any other requests
until the sleep function ended.

Clearly, I must be missing something; I have designed my application
upon the wrong axis. If so, I have some misunderstanding as to how to
properly structure an application like this upon Twisted. Or, twisted
is a framework whereby the response to a network event is expect to
arrive immediately.

I have seen "deferToThread" mentioned but I cannot find enough
documentation to decide whether it's what I need or not. Or perhaps
callInThread() is what I need?

Any suggestions about this would be very much appreciated. Even
better, if anybody knows of good documentation/tutorials they can
point me to, that would be appreciated also.

Regards,
Paul