[Twisted-Python] Twisted, Threading, Extension Modules and th e GIL

[Grant McDonald]

I just realised after re-reading my post that the call to deferToThread
should have been:
 
d = threads.deferToThread(self.builder.newbuild, buildJob)
d.addCallback(self.builderFinished)

How silly of me, i'll test it now and see if there are still problems!

-----Original Message-----
From: twisted-python-bounces at twistedmatrix.com
[mailto:twisted-python-bounces at twistedmatrix.com]On Behalf Of Grant McDonald
Sent: 04 March 2005 09:20
To: twisted-python at twistedmatrix.com
Subject: [Twisted-Python] Twisted, Threading, Extension Modules and the GIL



Hi everyone, 

I have a threading question regarding Twisted and the explanation is a bit
long-winded but please bear with me. 

The scenario: 

I have created an automated build system in Python for PowerBuilder (awful
language) that allows a scheduler to assign build jobs to dispatcher
programs across a network.  These dispatchers have a GUI component coded in
wxPython.  The initial implementation involved also creating a separate
builder program that was spawned by the dispatchers when a build job was
received.  The builder code called an extension module to interface with the
low-level PowerBuilder compiler library.  This was done due to the fact the
calls to rebuild the application and deploy it are exceptionally
CPU-intensive and don't return for many minutes (approx. 30 or so for a
large application) and all attempts to include this in the dispatcher
resulted in the user interface freezing.

Recently I came across twisted documentation with regards to programming
using twisted threads.  As an experiment on this concept I amalgamated the
dispatcher and 

builder programs and called the builder code using the following: 

d = threads.deferToThread(self.builder.newbuild(buildJob)) 
d.addCallback(self.builderFinished) 

It should be noted that the extension module (a pyrex extension I have
created myself) implements the releasing and re-acquiring of the Global
Interpreter Lock for all blocking calls.  From my understanding this should
work but the user interface freezes.  Can someone please point me in the
direction of a solution?

Thanks, 

Grant McDonald 

Important notice: This message is intended for the individual(s) and
entity(s) addressed. The information contained in this transmission and any
attached, may be confidential and may also be the subject of legal
privilege, public interest immunity or legal professional privilege. Any
review, retransmission, dissemination or other use of, taking of any action
in reliance upon this information by person or entities other than the
recipient is prohibited and requires authorization from the sender. If you
are not the addressee indicated in this message (or responsible for delivery
of the message to such person) you may not copy or deliver this message to
anyone. In such cases you should destroy this message and kindly notify the
sender by reply email. 

WARNING: Although Infocomp has taken reasonable precautions so that no
viruses are present in this e-mail, the company cannot accept responsibility
for any loss or damage arising from the use of e-mail attachments.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20050304/04d6379e/attachment.html>