[Twisted-Python] OT - adbapi, connection timeouts, mysql - OT
exarkun at twistedmatrix.com
exarkun at twistedmatrix.com
Thu Aug 27 07:46:00 MDT 2009
On 26 Aug, 11:08 pm, clay.gerrard at rackspace.com wrote:
>The way that sqlalchemy implements pool_recycle is "irrespective of
>idle time" best I can tell from looking at it. Keeping track of the
>last_used_time seems like it would have been equally reasonable,
>although perhaps even additional overhead.
It probably wouldn't be much more overhead. The important question to
consider is whether SQLAlchemy (and whoever else) had another good
reason for doing it the way they do it. Is monitoring idle time not
sufficient for some systems?
>
>I'm not sure I understand what you mean by "parameterize the reactor" -
>pass it as an argument when you initialize the (Re)ConnectionPool? Can
>you point me to a twisted class that does this as an example?
Yep, that's what I meant. You can find examples of this in some of the
newer code in Twisted (as we only started to encourage it relatively
recently). One example is twisted.web.wsgi.WSGIResource; another is
twisted.internet.task.deferLater.
>
>I agree it would be very cool "to do the reconnecting more proactively
>in idle threads" - I'll have to think on that, do you know off-hand of
>any analogs?
Hm. I can't think of any analogs, no. I hope the implementation would
be straightforward, though. The connection pool can use
reactor.callLater in the main thread to force old connections to be re-
established, and it can take those connections out of the lineup while
it's doing that, to make sure no queries are dispatched to them. One
tricky part, though, may be ensuring that a connection isn't actively
servicing a query when time comes to recycle it. The thread pool used
by the connection pool may not currently have the features required for
this (it doesn't allow messages to be directed at specific threads, for
example).
This feature may also not be terribly important. I don't know what the
reconnect overhead actually is.
Jean-Paul
More information about the Twisted-Python
mailing list