[Twisted-Python] client connecting to 2 servers (nonsimultaneously)

[exarkun at twistedmatrix.com]

On 06:45 pm, jrennie at gmail.com wrote:
>On Wed, Jan 12, 2011 at 11:32 AM, Glyph Lefkowitz
><glyph at twistedmatrix.com>wrote:
>>The reactor doesn't have a queue of tasks to be completed.  It has 
>>sets of
>>various event sources, which it executes in no particular order.
>
>"queue of tasks" was a guess on my part, but I looked through the
>BaseReactor code and found something like that.  'course, I could 
>easily be
>reading the code wrong.  How would you describe threadCallQueue?  When 
>I
>said "tasks", I meant something akin to "calls".  That could be
>confusing---did you interpret "tasks" differently?
>>Scheduling a timed event with callLater(0,...) might do what you want,
>>though.
>
>Yes.  Thanks!  I see that the callFromThread documentation even 
>recommends
>using callLater for this behavior. (doh!)
>
>http://twistedmatrix.com/documents/10.2.0/api/twisted.internet.interfaces.IReactorThreads.html#callFromThread
>>>>from twisted.internet import reactor
>>>>def foo():
>...     print "foo!"
>...
>>>>def bar():
>...     reactor.callLater(0, foo)
>...     print "bar!"
>...
>>>>reactor.callWhenRunning(bar)
>('startup', ('after', <function bar at 0x2c84320>, (), {}))
>>>>reactor.run()
>bar!
>foo!
>
>Interesting that you can substitute callFromThread for callLater(0, 
>...) in
>the above code and get the same behavior...
>
>Sorry to prolong the tangent, but I'd like to better understand the
>differences between callWhenRunning, callFromThread and callLater.  I 
>think
>reactor.wakeUp() is the one missing piece for me.  How exactly does it 
>work?
>Reading the BaseReactor code... callFromThread adds the call to a 
>queue,
>then calls wakeUp.  IIUC, wakeUp simply runs
>
>self.port.postEvent(0, KEY_WAKEUP, None)

Some reactors have a different implementation of wakeUp than the above, 
but they all have the same goal.

The reactor thread may be doing something which is going to block 
indefinitely (WaitForMultipleObjects, select(), epoll_wait(), etc).  The 
purpose of the wakeUp call is to cause that blocking call to end.  Once 
it ends, calls in the threadCallQueue can be processed.

The reactor is written such that if you are running code in the same 
thread it is running in (ie, you are "in the reactor thread") then any 
event source you create (be it network or time or whatever), the reactor 
will be sure to service it in a timely manner.  For network event 
sources, this means it will include a descriptor in the select() (etc) 
call.  For timing event sources (ie callLater), it means the reactor 
will set a timeout on the select() (etc) call so that it returns before 
it is time for that delayed call to execute.

However, if you are not in the reactor thread, then really the only 
thing you're allowed to do to Twisted is use reactor.callFromThread. 
Since this might happen at any time with respect to what the reactor 
thread is doing, there's no way to be sure a (let's call it a) thread 
call event will get handled in a timely fashion.  So this is the problem 
that wakeUp solves.  After the thread call event is added to 
threadCallQueue, wakeUp makes some event source the reactor is 
monitoring signal readiness (on POSIX, it writes a byte to a pipe the 
reactor is select()ing (etc) on; on Windows, it posts an event like you 
pasted above).  This ensures that if the reactor is asleep waiting for 
an event it will wake up soon, and then it will notice there are thread 
call events to process.

Jean-Paul