[Twisted-Python] defer.inlineCallbacks dosn't work with cython, deferredGenerator does.

[Glyph Lefkowitz]

On Apr 9, 2012, at 9:50 PM, Andrew Bennetts wrote:

> On Tue, Apr 10, 2012 at 12:41:17PM +0800, gelin yan wrote:
> […]
>>    When trying to run inlineCallbacks with Cython, a TypeError would come
>> out like:
>> 
>> TypeError: inlineCallbacks requires <built-in function test> to produce a
>> generator; instead got <cy3.__pyx_scope_struct__test object at 0x02808848)
>> 
>> test is the function where inlineCallbacks uses.
>> 
>> It looks like cython changes the name test to  cy3.__pyx_scope_struct__test
>> internally so generator can't work properly.
> 
> inlineCallbacks doesn't care what the function name is.  The problem here is
> that inlineCallbacks expects to be passed a generator function[1], and instead
> it is being passed some other sort of object.  The ugly cy3.__blah name is a red
> herring.
> 
> -Andrew.
> 
> [1] Without the check that produces this error, a common and very confusing
>    error is to decorate a non-generator function (i.e. one with no yield
>    statement) with @inlineCallbacks, which would appear to work but would then
>    mysteriously fail to do anything useful when executed.

Aah, I see.  So Twisted's check is a bit overzealous in the context of Cython.

Could you perhaps wrap the Cython pseudo-generator in a function that converts it into something like a real Python generator, or disable the warning and see if it works as expected?  PEP 380 <http://www.python.org/dev/peps/pep-0380/> does have some code which one might use to do general-purpose iterator-to-generator conversion.

In any case, please file a bug.  Search the tracker for duplicates first, of course, but there don't appear to be any.  There is some precedent for fixing this type of error, too: <http://twistedmatrix.com/trac/ticket/3132> was somewhat similar.

-glyph