[Twisted-Python] Connection poolin

[Tommi Virtanen]

On Sat, Jul 26, 2003 at 08:38:30PM -0500, John Goerzen wrote:
> imap = self.imapserver.acquireconnection()
> try:
>     # select the appropriate mailbox
>     # do stuff here
>     return result
> finally:
>    self.imapserver.release(imap)

	Here's two quick hacks that differ mostly in where you put
	the connection release, and how you organize the deferreds.

	Method 1: manage the release in the caller, but need to jump
	through hoops to get the connection passed to
	_releaseConnection. self.imapserver.acquireconnection() is
	of the traditional "returns a Deferred" style.

class Something:
    def startDoingStuff(self):
	d = self.imapserver.acquireconnection()
	d.addCallback(self._gotConnection)
	return d

    def _gotConnection(self, connection):
	d = defer.Deferred()
	d.addCallback(self._doStuff)
	d.addBoth(self._releaseConnection, connection)
	d.callback(connection)
	return d

    def _doStuff(self, connection):
	# select the appropriate mailbox
	# do stuff here

    def _releaseConnection(self, stuff, connection):
	connection.release()
	return stuff


	Method 2: manage the release in the connection pool. Needs
	to change the API of acquireconnection into taking a deferred,
	as otherwise it would be impossible to make _doStuff get called
	before the connection release. Note how close this is to the above,
	with just release logic located in the same class as
	acquireconnection.

class SomethingBetter:
    def startDoingStuff(self):
	d = defer.Deferred()
	d.addCallback(self._doStuff)
	self.imapserver.acquireconnection(d)
	return d

    def _doStuff(self, connection):
	# select the appropriate mailbox
	# do stuff here

class ...:
    def acquireconnection(self, d):
	d2 = self._grabConnection()
	d2.addCallback(self._gotConnection, d)
	d2.chainCallback(d)

    def _gotConnection(self, connection, d):
	d.addBoth(self._releaseConnection, connection)
	return connection

    def _releaseConnection(self, stuff, connection):
	connection.release()
	return stuff

> I've thought that in each method I have that needs a connection, I
> could write two methods: one that calls the Twisted version of
> acquireconnection and sets its callback to the second, which does the
> real work (and possibly chains to others via callbacks.)
> 
> There are some problems with that approach.  One is that it is very
> verbose and requires a lot of code all over the place to manage.
> Another is that there's no easy way to tell in Twisted when we are
> done with the given connection.  I suppose I could always add errbacks
> all along the way plus a callback, each of which releases the
> connection, but that seems inconvenient and error-prone.

	I don't think the method 2 above is verbose. And there is no reason
	to add errbacks "all along the way". Just a single addBoth is
	enough. When the .addBoth callback returns it's first arg, it
	pretty much equals python exception handling try: ... finally: ...

-- 
:(){ :|:&};: