[Twisted-Python] Twisted, Threading, Extension Modules and the GIL

[Grant McDonald]

Hi everyone,

I have a threading question regarding Twisted and the explanation is a bit
long-winded but please bear with me.

The scenario:

I have created an automated build system in Python for PowerBuilder (awful
language) that allows a scheduler to assign build jobs to dispatcher
programs across a network.  These dispatchers have a GUI component coded in
wxPython.  The initial implementation involved also creating a separate
builder program that was spawned by the dispatchers when a build job was
received.  The builder code called an extension module to interface with the
low-level PowerBuilder compiler library.  This was done due to the fact the
calls to rebuild the application and deploy it are exceptionally
CPU-intensive and don't return for many minutes (approx. 30 or so for a
large application) and all attempts to include this in the dispatcher
resulted in the user interface freezing.

Recently I came across twisted documentation with regards to programming
using twisted threads.  As an experiment on this concept I amalgamated the
dispatcher and 
builder programs and called the builder code using the following:

d = threads.deferToThread(self.builder.newbuild(buildJob))
d.addCallback(self.builderFinished)

It should be noted that the extension module (a pyrex extension I have
created myself) implements the releasing and re-acquiring of the Global
Interpreter Lock for all blocking calls.  From my understanding this should
work but the user interface freezes.  Can someone please point me in the
direction of a solution?

Thanks,

Grant McDonald

-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20050304/58583981/attachment.html>