[Twisted-Python] Re: Other ways to integrate foreign event loops?

[David Bolen]

Nathan <nathan.stocks at gmail.com> writes:

> On Mon, Mar 31, 2008 at 8:17 PM,  <glyph at divmod.com> wrote:
>>  Twisted's job is to run the main loop; it's best just to let it do that.
>>  If Pyglet slowed down, it would be best to examine what's gone wrong
>>  with Pyglet and simply submit a fix for that.
>
> Understood.  Thanks for the guidance.  I'm working on example code
> that reproduces the problem.

While in general I'm always in favor of letting Twisted own the main
loop, I've also had quite good success in the past not doing so, even
without multiple threads, primarily in cases where networking was a
secondary function (e.g., GUI applications) and less time crucial than
the primary UI interaction.

In our case, with wxPython based applications, things worked great
just having wxPython handle a timer that triggered reactor iterations.
It involved:

* During initialization, start reactor with reactor.startRunning()
* At some periodic frequency, run reactor.iterate(0)
* During termination, set up a reactor.callLater for reactor.stop, and
  then call reactor.mainLoop().  This was the cleanest way we found to
  permit all reactor finalization (events, final I/O, etc...).

Note that the first point violates the IReactorCore interface
(startRunning() should only be accessed through run()).  I don't
consider running mainLoop() at the end a violation since it's the only
time it is called.

Yes, the above means that Twisted's loop might incur higher latency
(we ran the timer at about 150ms) than it might be if in control, but
it worked very well for us, including during times when the
win32/wxPython reactors weren't particularly stable, and
_threadedselect wasn't yet an option.  And even if in control, once
Twisted calls out to pump GUI messages, large latency is still
theoretically possible until it regains control.

So while I wouldn't usually suggest making Twisted's main loop
subservient to some other loop (and trying to identify the issue in
Pyglet would be a good thing), it's also something I'm not averse to
doing when needed.

It might be worth experimenting with using Pyglet's main loop, and
pyglet.clock.schedule or pyglet.clock.schedule_interval to iterate
Twisted's reactor.

-- David