[Twisted-Python] shared list in factory, mutexes?

[Gabriel Rossetti]

Itamar Shtull-Trauring wrote:
> On Mon, 2008-04-14 at 12:35 +0200, Gabriel Rossetti wrote:
>   
>> I'm wondering if maybe I'm misunderstanding something here, when several 
>> messages arrive, what happens, are they processed in parallel, are they 
>> put in a queue and processes sequentially? 
>>     
>
> Sequentially. Twisted does not expose use of threads to user-level
> APIs... and Twisted APIs are not thread-safe, other than
> reactor.callFromThread.
>
>   
>> If I receive a message, do 
>> some heavy processing/io/potentially blocking code in a thread using 
>> twisted's threading module, and before it's done, I get another 
>> connection + message that needs the same thing to be done, what
>> happens? 
>>     
>
> reactor.callInThread (and therefore deferToThread) are built on top of a
> thread-safe Queue. Just make sure your scheduled tasks are not calling
> back into Twisted but are rather self-contained units of work.
>
>   
Ok, but if connection1 comes, it does some processing by using a thread 
or a process (with twisted's functions) and connection2 comes, will 
connection2 wait for connection1 to finish, or will connection1 be put 
on the side while it waits for the thread and connection 2 is taken. 
Also, what event makes the next connection be

 think what I don't get is that with thread network code, each thread is 
an individual self-contained unit that does whatever processing it needs 
to do however long it may need to take, and the OS takes care of 
scheduling each thread, but with twisted, the main thread executes each 
connection one by one sequentially and thus if one of them needs to to 
some lengthy processing, it will block all other connections from being 
processed, so you have to use threads to keep it from blocking, but 
somehow the active connection should become inactive while the 
processing thread does it's work, so that another connection can be 
processed in the meantime. Is that how it works? So twisted's event loop 
(reactor) does the same work as the OS's scheduler?

Thanks