[Twisted-Python] Why do I get an AlreadyCalledError?

[Jean-Paul Calderone]

On Sat, 28 Feb 2009 13:21:06 +0000, Conrad Winchester <conradwinchester at me.com> wrote:
>Hi all,
>
>I  am writing a restful twisted (8.1.0) server and am trying to return 
>useful error statuses from requests. The problem is that even  following the 
>documentation extremely carefully I am getting an  AlreadyCalled Error every 
>time I generate a failure. Here is my main  class (leaving out code that I 
>feel is not relevant to the question:
>
>from xml.dom import minidom
>from twisted.web import server, resource
>
>from FWGException import FWGException
>
>class DeferredPostHandler(resource.Resource,minidom.Document):
>
>     NOT_IMPLEMENTED = 501
>     BAD_REQUEST = 400
>
>     def writeError(self, failure, request, *args, **kw):
>         failure.trap(FWGException)
>         request.setResponseCode(self.errorCode)
>         request.write("Error handling request: %s" % 
>failure.getErrorMessage())
>         request.finish()
>
>     def writeResult(self, result, request, *args, **kw):
>         self.writexml(request)
>         request.finish()
>
>     def render_GET(self, request):
>         self.process=self.db.runInteraction(self.processForm,request)

I think that self.db is a ConnectionPool instance.  This means that here,
you're starting self.processForm in a thread.  I'm not sure if this is
the reason you're having difficulties, but it may be.  See below.

>         self.process.addCallbacks(self.writeResult, self.writeError, 
>[request], {}, [request], {})
>         return server.NOT_DONE_YET
>
>
>I have a handler that extends this
>
>from DeferredPostHandler import DeferredPostHandler
>from twisted.web import http
>from twisted.python.failure import Failure
>from FWGException import FWGException
>
>class CollectionAdminHandler(DeferredPostHandler):
>
>     def __init__(self, task, db, config):
>         DeferredPostHandler.__init__(self,db)
>         self.config = config
>         self.task = task
>         self.errorCode = http.OK
>
>     def processForm(self, cursor, request):
>
>         if self.task == 'add':
>             self.addCollection(cursor, request)
>
>         elif self.task == 'delete':
>             self.deleteCollection(cursor, request)
>
>         else:
>             self.errorCode = DeferredPostHandler.NOT_IMPLEMENTED
>             fail = Failure(FWGException("Unsupported Task"))
>             self.process.errback(fail)

Here you're creating a Failure and errbacking a Deferred in a thread, since
processForm is running in a thread.  This isn't allowed.  Twisted APIs are
generally not safe to be called in any thread except the reactor thread.

>
>     def addCollection(self, cursor, request):
>         fail = None
>         if self.isValidAddRequest(request):
>             pass
>         else:
>             self.errorCode = DeferredPostHandler.BAD_REQUEST
>             fail = Failure(FWGException("Missing Parameter"))
>
>         if fail:
>             self.process.errback(fail);

Likewise here.

>
>     def deleteCollection(self, cursor, request):
>         print 'Delete Collection'
>
>     def isValidAddRequest(self, request):
>         return 'title' in request.args and 'category' in request.args  and 
>'date' in request.args
>
>
>Unfortunately whenever I call
>
>
>self.process.errback(fail)
>

To remove the incorrect calls to Twisted APIs from a thread, you should
probably just raise an exception where you're currently errbacking that
Deferred.  In render_GET, or in writeError, you can handle the failure
which will result from that.

Ah, and now I see your problem.  self.process is the Deferred returned
by runInteraction.  You're not supposed to errback that Deferred (or
callback it either).  A pretty good rule of thumb is that if you create
a Deferred, then you get to fire it.  In this case, runInteraction is
creating it, so runInteraction gets to fire it.  runInteraction will
fire it with the result of processForm, so that's why you get the error.
You're firing that Deferred and runInteraction is firing it.

Jean-Paul