[Twisted-Python] pb help needed.

[Syver Enstad]

Syver Enstad <syver at inout.no> writes:

> Syver Enstad <syver at inout.no> writes:
> 
> > I have a bug in my woven application which manifests itself as
> > win32select failing after doing many http requests. The error is the
> > constant WSAENOBUFS and I it seems pretty likely that this is caused
> > by some socket related resource leaking for each request I make. This
> > hypothesis is further supported by observing that a host of
> > pb.ClientFactory instances are stopped when shutting down the
> > application after it crashes. This seems strange to me as I am
> > thorough in shutting down the connector after every request is
> > finished.
> > 
> > Turning to that trusty friend printf style debugging, shows that the
> > PBClientFactory is stopped correctly when performing requests to pages
> > that use the referenceable and wait for it's deferred's to
> > complete. There are two cases when it doesn't stop. 
> > 
> > The first is when I press reload quickly a number of times. Then I
> > just see start messages because my request finish method is not called
> > (it is a callback to the deferred returned by request.notifyFinish()).
> > 
> > The second case is when I call disconnect on the connector before it's
> > deferred has been realised. I see in the log the following pattern:
> > Starting pb.PBClientFactory, disconnect and then the attached event,
> > stopping factory is not called at all. After stepping I see that
> > disconnect doesn't do anything when it is called before the connection
> > has been realized.
> > 
> > So I have a bug, and it's in my code. My question is how does one
> > manage pb resources correctly and easily (I don't worry about
> > performance just yet) when each user needs to have it's own
> > connection?
> 
> Here is what I have come up with so far. It doesn't solve the problem
> where I don't get the request finish callback, but seems to be able to
> disconnect even if not connected when disconnect is called.
> 
> class MyPBClientFactory(pb.PBClientFactory):
>     def disconnect(self):
>         if not self._broker:
>             deferred = self.getRootObject()
>             deferred.addCallback(
>                 lambda unused: pb.PBClientFactory.disconnect(self))
>         else:
>             pb.PBClientFactory.disconnect(self)

Since the last post I moved to a persistent PBClientFactory so that
disconnects is not a problem. This fixes the crashes, but there are
still resources that are not released. I only see perspective attached
messages in the log, until I kill twistd when I get a host of detached
messages. My next question must be how do I detach the perspectives in
an orderly fashion after each request? This seems to run into the same
problems as I had with not being able to do cleanup for each request.