[Twisted-Python] self.channel.writeHeaders(version, code, reason, headers)\nbuiltins.AttributeError: \'NoneType\' object has no attribute \'writeHeaders\'\

[Waqar Khan]

Hi Glyph,
   Here is the minimal version


class FooResource(resource.Resource):
       def render_GET(request):
              future = asyncio.ensure_future(self.fetch_response(request))
// some async await functions
              d = Deferred.fromFuture(future)
              d.addCallback(print_json_response, request) // this is
actually where the error is triggered.
              d.addErrback(lambda failure:
failure.trap(defer.CancelledError))
              finished_errback = request.notifyFinish()
              finished_errback.addErrback(self.handle_cancel, d) // simple
logs and cancels d by d.cancel()
              return NOT_DONE_YET

The traceback is like
'Traceback (most recent call last):\n  File
"/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/events.py",
line 88, in _run\n    self._context.run(self._callback, *self._args)\n
 File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py",
line 827, in adapt\n    adapt.actual.callback(extracted)\n  File
"/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line
460, in callback\n    self._startRunCallbacks(result)\n  File
"/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line
568, in _startRunCallbacks\n    self._runCallbacks()\n--- <exception caught
here> ---\n  File
"/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line
654, in _runCallbacks\n    current.result = callback(current.result, *args,
**kw)\n  File "/Users/wqKhan/twisted_eg/parse_response.py", line 3, in
print_json_response\n    *print_json_response*(response, request)\n  File
"/Users/wqKhan/twisted_eg/parse_response.py", line 21, in
*print_json_response*\n    request.finish()\n  File
"/usr/local/lib/python3.7/site-packages/twisted/web/server.py", line 268,
in finish\n    return http.Request.finish(self)\n  File
"/usr/local/lib/python3.7/site-packages/twisted/web/http.py", line 1071, in
finish\n    "Request.finish called on a request after its connection was
lost; "\nbuiltins.RuntimeError: Request.finish called on a request after
its connection was lost; use Request.notifyFinish to keep track of this.\n'



On Sun, Aug 4, 2019 at 10:36 PM Glyph <glyph at twistedmatrix.com> wrote:

>
>
> > On Aug 4, 2019, at 5:57 PM, Waqar Khan <wk80333 at gmail.com> wrote:
> >
> > Hi Glyph,
> >   Thanks for the suggestion.
> > I tried the suggestion.. While it fixes the self.channel NoneType
> issue.. It creates another issue.
> >
> > Traceback (most recent call last):
> >   File
> "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/events.py",
> line 88, in _run
> >     self._context.run(self._callback, *self._args)
> >   File
> "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line
> 827, in adapt
> >     adapt.actual.callback(extracted)
> >   File
> "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line
> 460, in callback
> >     self._startRunCallbacks(result)
> >   File
> "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line
> 568, in _startRunCallbacks
> >     self._runCallbacks()
> > --- <exception caught here> ---
> >   File
> "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line
> 654, in _runCallbacks
> >     current.result = callback(current.result, *args, **kw)
> > ......
> >     request.finish()
> >   File "/usr/local/lib/python3.7/site-packages/twisted/web/server.py",
> line 268, in finish
> >     return http.Request.finish(self)
> >   File "/usr/local/lib/python3.7/site-packages/twisted/web/http.py",
> line 1071, in finish
> >     "Request.finish called on a request after its connection was lost; "
> > builtins.RuntimeError: Request.finish called on a request after its
> connection was lost; use Request.notifyFinish to keep track of this.
>
> Oh, hrm.  The idea here is that any code that returns NOT_DONE_YET (i.e.
> does asynchronous work to generate a response) should also be tracking
> whether it needs to call `.finish()` by watching a `.notifyFinish()`
> Deferred.  I bet that there are plenty of resources in Twisted which don't
> follow this rule, and we should fix those; but possibly we should also make
> this error message less stern.  Do you have a minimal example?
>
> >  I have got some handle on the issue. I was wondering if you have any
> advise on how to debug the issue.
> > Somewhere in the codebase are async/await code (from asyncio world).
> Hence, when the connection is closed down by client, seems like those
> pending coroutines are lingering on.
>
> This, I have no idea about.  What tasks are you starting, when are you
> expecting them to get cleaned up, what are they blocking on, how do these
> interact with Twisted?  There are a couple dozen questions I'd have to know
> the answer to in order to even begin debugging this.  If you cancel all the
> outstanding tasks and look at their tracebacks when exiting it might give
> you more of a sense of where they're stuck...
>
> -g
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20190804/8cb46f00/attachment-0002.html>