[Twisted-Python] Deferred usage clarification

[Andrew Bennetts]

On Sun, Jun 01, 2003 at 09:27:09PM -0400, Gary Poster wrote:
> Andrew Bennetts wrote:
> >You're misunderstanding the purpose of Deferreds.  They don't do any magic
> >to make blocking calls happen asynchronously.
> ...snip...
> >
> >Does this make sense?
> 
> Yes.  Looks like deferToThread would be the simplest way to do what I 
> want.  I wonder if you (or someone) would answer a similarly newbie-like 
> follow up, though.
> 
> I see the pattern
>   reactor.callLater(0, function)
> a lot--this doesn't have thread semantics, though, so what is 
> accomplished by specifying the 0 second delay?

This one's easy ;)

That idiom just runs that function in the next iteration of the event loop
-- so more events (more network traffic, or other delayed calls, gui events,
or whatever) can be processed immediately if necessary.

A single reactor.callLater(0, ...) won't make a big improvement to the
responsiveness of the program -- delaying a 3-second blocking call still
blocks the program for 3 seconds.  But if you can break up a long-running
calculation into smaller pieces, then it can help alot.  Imagine you just
suddenly received 20000 rows of data from a database; processing that all in
one hit will probably take a noticeable amount of time, even if that
processing is trivial.  But if you break up the processing into chunks of,
say, 100 rows at a time, your app will be much more responsive to external
events while still getting the work done.

Another use for postponing something to the next iteration of the event-loop
is simply to ensure that the function doesn't run until something else has
happened in *this* iteration.  Using it like this is fairly subtle; you'd be
well-advised to comment thoroughly what you're doing if you do this.

-Andrew.