[Twisted-Python] Synchronous Wrapper for Deferreds

[glyph at divmod.com]

On Fri, 18 Mar 2005 23:13:22 -0600, Ken Kinder <ken at kenkinder.com> wrote:
>A day or two ago, I inquired on IRC about making calls from an existing 
> synchronous script. The script is not related to any Twisted activity, 
> so minimal intervention was sort of a goal. Obviously this doesn't work 
> on non-transient objects such as Perspective Broker ones, but here's 
> what I came up with... (See attached) Obviously there is some cleaning 
> up to do, but it demonstrates the idea.

I would suggest starting (using os.popen or suchlike) a separate python script which uses Twisted to generate the output that you require to standard output.

> I have three questions: (1) I know this isn't the ideal way to use 
> Twisted, but is there anything inherently problematic about my solution 
> for one-off calls to Twisted.

It's possible, insofar as the unit tests already do it.

It's extremely undesireable, because only one piece of synchronous code can be "driving" at a time.  Once you decide you'd want to integrate synchronous twisted-using code A with synchronous twisted-using code B, you're screwed. By contrast, any number of asynchronous pieces of code can be used in the same program.

> (2) I understand there are problems with 
> running the same reactor twice. What are those problems and is there a 
> recommended work around?

The problems boil down to this: the behavior is more or less undefined.  The recommended workaround is to fix about 30 or 40 extremely subtle bugs in various implementations of the reactor, define a sane behavior, and write unit tests to make sure that it is scrupulously followed by all implementations.  Sorry :).

The good news is that these bugs are almost entirely in the realm of service startup and shutdown.  If you're not using any Service objects, you're likely to run into relatively few problems.

> (3) Are there any ideas on making a synchronous 
> version of a Perspective Broker client to integrate into existing 
> synchronous applications more easily?

Unfortunately PB is inherently both asynchronous and symmetrical and either side can submit method calls to any available object at any time.  A synchronous frontend would make this impossible, and would produce some very confusing race-conditions in relatively normal PB code.

You might be able to specify a subset of PB as a strict request/response protocol with no reentrancy, but why bother?  If you're doing synchronous scripting I imagine it would be easier to add an XMLRPC frontend (which works fine in a synchronous context) to your Twisted server than to connect over PB and have these kind of integration headaches.