[Twisted-Python] A kinder and more consistent defer.inlineCallbacks
Drew Smathers
drew.smathers at gmail.com
Fri Nov 21 12:20:33 MST 2008
On Fri, Nov 21, 2008 at 1:04 PM, Terry Jones <terry at jon.es> wrote:
> Here's a suggestion for making inlineCallbacks more consistent and less
> confusing. Let's suppose you're writing something like this:
>
> @inlineCallbacks
> def func():
> ....
>
> result = func()
>
> There are 2 things that could be better, IMO:
>
> 1. func may not yield. In that case, you either get an AttributeError when
> inlineCallbacks tries to send(). Or worse, the call to send might actually
> work, and do who knows what. I.e., func() could return an object with a
> send method but which is not a generator. For some fun, run some code that
> calls the following decorated function (see if you can figure out what will
> happen before you do):
>
> @defer.inlineCallbacks
> def f():
> class yes():
> def send(x, y):
> print 'yes'
> # accidentally_destroy_the_universe_too()
> return yes()
>
Why not just return a Deferred from the function and not decorate it.
Or document the function as returning a value if it doesn't block.
> 2. func might raise before it get to its first yield. In that case you'll
> get an exception thrown when the inlineCallbacks decorator tries to create
> the wrapper function:
>
> File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 813, in unwindGenerator
> return _inlineCallbacks(None, f(*args, **kwargs), Deferred())
>
>
> There's a simple and consistent way to handle both of these. Just have
> inlineCallbacks do some initial work based on what it has been passed:
>
> def altInlineCallbacks(f):
> def unwindGenerator(*args, **kwargs):
> deferred = defer.Deferred()
> try:
> result = f(*args, **kwargs)
> except Exception, e:
> deferred.errback(e)
> return deferred
> if isinstance(result, types.GeneratorType):
> return defer._inlineCallbacks(None, result, deferred)
> deferred.callback(result)
> return deferred
>
> return mergeFunctionMetadata(f, unwindGenerator)
Essentially this is equivalent to _not_ decorating your function and
returning a Deferred via defer.succeed(value) or defer.fail(error).
I'm still not sure why it's necessary to stitch this kind of behavior
into inlineCallbacks(). From my perspective inlineCallbacks() simply
equates: "I want to do more than one asynchronous operation inline and
here's some syntactic sugar using yield as an expression."
>
> This has the advantage that (barring e.g., a KeyboardInterrupt in the
> middle of things) you'll *always* get a deferred back when you call an
> inlineCallbacks decorated function. That deferred might have already called
> or erred back (corresponding to cases 1 and 2 above).
>
You will always get a Deferred back if the function is a generator.
> I'm going to use this version of inlineCallbacks in my code. There's a
> case for it making it into Twisted itself: inlinecallbacks is already
> cryptic enough in its operation that anything we can do to make its
> operation more uniform and less surprising, the better.
>
> You might think that case 1 rarely comes up. But I've hit it a few times,
> usually when commenting out sections of code for testing. If you
> accidentally comment out the last yield in func, it no longer returns a
> generator and that causes a different error.
That's what unit tests are for :) But I'm not sure how you could
easily wind up in this scenario considering generators don't allow
return with an argument.
Drew
More information about the Twisted-Python
mailing list