[Twisted-Python] [Fwd: Re: Deferred documentation rewrite]

[Edward Z. Yang]

Excerpts from Itamar Shtull-Trauring's message of Fri Jul 31 22:26:29 -0400 2009:
> The problem with this is that it perpetuates the misunderstanding the
> Deferreds *make* things asynchronous, even with the intro that says
> otherwise. I think it's better to assume already asynchronous code,
> handling the transition from synchronous to async in an intro event loop
> howto. 

Either way, the function that the first segment of the new docs do belong
somewhere.  The documentation that traditionally served this purpose
has been removed.

As for perpetuating the misunderstanding of Deferreds making things
asynchronous, I completely agree!  However, I think this is something
that can be fixed by spelling out the distinction between "writing
asynchronous code" and "interacting with asynchronous code", and not
just omitting the important paradigm shift that comes with sync->async.

> A better comparative exposition might be with normal callbacks, e.g.:
> 
> "def foo(x, gotResultCallback): pass" vs. "def foo(x): # return
> Deferred".
> 
> At the very least having that async but callbacky version in the middle
> helps understanding.

I briefly gloss on this, but I agree that this is an important point
that could be further expanded.  We could have implemented asynchronous
mechanisms using normal callbacks, but we decided to use Deferreds instead.

It's not clear to me if the common case of confusion of Deferreds occurs
in people who "know callbacks" but "don't know Deferreds."  As an incoming
developer who was familiar with asynchronous programming, my primary problem
was the ill-defined behavior of callback chains (which I resolved by
hunkering down and reading the source code) rather than any fundamental
misunderstanding of what Deferreds were supposed to do.

> It also omits half the story: how you *create* Deferreds. There should
> be a section on that as well.

I agree.  In fact, it might be worth making the document a little longer
to address this point, because I realize now that even if you're not
writing asynchronous code, you'll often need to baton Deferreds to
make the execution flow work the way you want them to.

> An example involving a parser, where you just wave your hands about who
> pushes data in to the parser exactly (so no need to go into event loop
> details), may work well. In particular, the object that wants the result
> of the parsing wants to get parse errors, *not* whoever pushes data in.
> Often it's same object, but not always. Deferreds help with that.

That's a good distinction.  I'll see how I can work that in.

Cheers,
Edward