[Twisted-Python] Synchronization techniques

[Ed Suominen]

Daniel Miller wrote:
> I'm sorry I wrote that...it was inflammatory and did not bring any value
> to the conversation. Please accept my apology.

That really shows some class. Thanks for saying it.

>> Another solution here would be for Twisted to have a nice convenience
>> API for dispatching tasks to a process pool.  Right now setting up a
>> process pool is conceptually easy but mechanically difficult; you have
>> to do a lot of typing and make a lot of irrelevant decisions (AMP or
>> PB or pickle?  stdio or sockets?).
> 
> That sounds nice.

There's been a lot of interest in this, including from Glyph Lefkowitz.
The taskqueue API in my twisted-goodies package could be easily adapted
to do it, using an implementation of IWorker that works with python
interpreter processes rather than threads. (See
http://foss.eepatents.com/api/twisted-goodies/taskqueue.workers.html.)
Note that you can queue to remote workers with the API just as it stands
now, see RemoteCallWorker and RemoteInterfaceWorker.

If anyone wants to take a shot at a process pool using a ProcessWorker,
I'll be happy to help. And if there's interest in having the result go
into Twisted proper under an MIT license, I remain open to that.

Best regards, Ed