[Twisted-Python] shared list in factory, mutexes?

[Simon Pickles]

Gabriel Rossetti wrote:
> each thread is an individual self-contained unit that does whatever 
> processing it needs to do however long it may need to take, and the OS 
> takes care of scheduling each thread, but with twisted, the main 
> thread executes each connection one by one sequentially and thus if 
> one of them needs to to some lengthy processing, it will block all 
> other connections from being processed, so you have to use threads to 
> keep it from blocking, but somehow the active connection should become 
> inactive while the processing thread does it's work, so that another 
> connection can be processed in the meantime.
>
>
Hi Gabriel,

Have you considered implementing Stackless python in partnership with 
Twisted? I am finding it very useful for handling multiple concurrent 
connections in a single thread. (it uses a scheduler which can switch 
between tasklets, in a yield fashion.)

The programmer has complete control over where this tasklet switching 
takes place, vastly simplifying things over multi-threading. If one 
connection involves some heavy processing, I simply add a 
stackless.schedule() in the blocking loop which allows waiting jobs to 
run too. Overhead for this tasklet switching is almost negligible too, 
unlike threads.

Regards

Simon