[Twisted-Python] Re: deferToThread - supported alternative to the (deprecated) setTimeout method
Paul Moore
pf_moore at yahoo.co.uk
Fri May 26 13:46:27 MDT 2006
Many thanks to all for the helpful responses. They've given me a lot
of options to think about. I'll see where it takes me.
Some specific comments:
On Tue, 23 May 2006 22:43:35 -0400, Jean-Paul Calderone
<exarkun at divmod.com> wrote:
>There's some code along these lines here:
>
>http://twistedmatrix.com/trac/browser/trunk/twisted/internet/base.py#L201
Thanks, that was useful.
>It might make a good example to work from. Note that any call that
>blocks indefinitely will hold on to a thread indefinitely, and thus
>reduce your effective thread pool size by one. If this happens
>enough, you'll end up with no free threads in your threadpool, and
>no other threaded tasks will ever be able to complete.
That's a good point. My initial thought was that I'm not writing a
long-running process here (my process runs all the threads, does a bit
of housekeeping when they complete, and then stops) and the blocking
thread condition is rare, so it shouldn't be a problem. But that's
just swapping one rare problem for another, somewhat rarer case. So
maybe I'm better fixing it properly.
On Wed, 24 May 2006 09:03:29 +1000, "Eric Mangold"
<teratorn at twistedmatrix.com> wrote:
>Well this is a bit of a problem. You can't just kill a thread - so it will
>eventually return a result, and Twisted will .callback() its deferred. If
>you've already stepped in, due to a timeout, and .callback()'ed the same
>deferred you'll get an AlreadyCalledError in your log.
Ah. That clarifies why setTimeout is deprecated, and why it's not
appropriate for me. Thanks!
On Wed, 24 May 2006 12:36:37 +1000, Andrew Bennetts
<andrew-twisted at puzzling.org> wrote:
>I don't suppose there's a non-blocking way to do what you want?
Sadly, no. It's a horribly broken API in this respect. (It's the
Oracle database connection API, if you want to know - no timeout
capability, and a rare but definite chance of a complete hang, no
return *ever*).
>Oh -- and remember that while you can't kill threads, you *can* kill processes.
>Consider using subprocesses to do your blocking work.
That's possible, but I'm running on Windows and subprocess management
is not as manageable on that platform (even under Python/twisted). But
it is a thought, and I'll keep it in mind.
Paul.
--
The trouble with being punctual is that nobody's there to appreciate
it. -- Franklin P. Jones
More information about the Twisted-Python
mailing list