[Twisted-Python] Multiple main loops

[Markus Schiltknecht]

Hi,

I'm not sure you are talking about the same topic. But I have just been
thinking about a threaded reactor.

Because my biggest itch with twisted is, that it can not really make use
multi-processor systems. Especially nowadays with dual-core or even
multi-core processors hitting the market this seems like a must-have.

In theory, it would be possible to start as many threads as you have
processors. All threads would process 'events' of the main loop. See for
example the apache event MPM for a better explaination of the algorithm.

Beside allowing only one thread at a time to call select(), this would
have other implications for twisted. I'm not sure about what they are,
but I can imagine it's a non-trivial goal.

How about todays python thread implementation? Did it improve over the
years? Is it worth using? What must be considered to get a reasonably
good performance using python threads?

..just some ideas. I'd be thankfull for hints, though.

Regards

Markus

On Tue, 2006-05-02 at 12:00 -0300, David Pratt wrote:
> Hi. I am hoping you can clarify for me whether it is possible to run 
> multiple independent main loops in Twisted. It is my understanding that 
> this would mean separate independent reactors.
> 
> Googling, I found this patch but after looking into the repository, it 
> was never committed. I would appreciate any comments on the patch or 
> attempts people have made on the integration of a second twisted loop.
> 
> At the present time I am using twisted with wxpython with wxpython as 
> main loop and also running a selectreactor instance. Here there are two 
> separate loops, interoperating and operating independently, however they 
> are not both twisted loops.
> 
> Zope currently runs using a reactor instance for each zope instance but 
> I would like to consider a separate independent loop for each client to 
> synchronize to a server without paying a penalty. Zope's ZEO (basically 
> a synchronization server) currently uses a simple zrpc protocol and runs 
> independently but interoperates with a separate ansyncore main loop. The 
> current system works well for the exception that it would be great to 
> simplify this as well as improve security using twisted's built in 
> capabilities. Many thanks.
> 
> 
> Regards
> David
> 
<SNIPPED the attached patch>