[Twisted-Python] Considering Twisted for OfflineIMAP

[Tommi Virtanen]

On Sat, Jul 26, 2003 at 08:48:15AM -0500, John Goerzen wrote:
> My first plan was to just rewrite imaplib, which wouldn't be too
> hard.  But someone on the OfflineIMAP list pointed me to Twisted,
> which I observe already contains an IMAP library that looks nice, plus
> a different paradigm that may spare me from the whole thread groups
> question altogether.

	Oooh! I'm very motivated to helping you; I've used offlineimap
	for a long time now, and I think I grok Twisted reasonably
	well.

	We're on IRC at #twisted on freenode. I'd be happy to help you
	while you are getting to know twisted.

	Andrew gave pretty good answers already, so I'll keep my
	answers short.

> 1. How does Twisted interact with things like threads and connection
>    pooling?

	You really don't want to use threads; Twisted gives you everything
	you need to implement offlineimap asynchronously.

	(There is e.g. deferToThread, used when interfacing to e.g. database
	APIs that block, but that really should not be used if possible.)

>    I currently let the user control the maximum number of connections
>    that are open to a given server, and whenever the code needs to
>    access an IMAP server, it grabs a connection from the appropriate
>    pool and uses it (using threading primitives to block if no
>    connection is available.)  Can Twisted handle multiple connections
>    to multiple servers doing multiple things at once?  Can it do
>    connection pools?  If it needs threading to do these things, does
>    it play nice with it?

	That should be pretty doable, not with thread pooling but by just
	counting the currently open IMAP connections and when the number
	is below the limit, taking more work from a queue and starting
	to process it (asynchronously).

> 4. The howto "book" alludes to pending improvements on the mail
>    infrastructure.  Anything I should be aware of here?

	I believe much of that has already happened.

> 5. Is there any sort of unified mail API in Twisted (like JavaMail)
>    that would present me with a single API to both IMAP and Maildir
>    repositories, or is that something I need to do on my own?  (I've
>    already done it, so it's no big deal to do that again)

	Well, see twisted.protocols.imap4.IMailbox for _something_.

> 6. OfflineIMAP supports several different user interfaces (two written
>    with Tkinter, 1 using Curses, and three plain console ones.)  I would
>    like to reimplement the Tkinter ones with wxPython, and am glad to
>    see that Twisted works well with this.
> 
>    One concern, though, is that the users can supply a list of UIs to try
>    in the config file: for instance, Tk, Curses, Noninteractive.  The
>    system will try each one in turn until it reaches one that works.
>    (For instance, Tk may not work if the user is not in X, and Curses
>    may not work if stdout is not a tty).  It looks like this may be
>    problematic with Twisted since the UI selection always seems to be
>    known in advance with the import commands related to the reactor.
>    Any thoughts here?

	Hmm. It seems you'd need to read the configuration before starting
	the reactor (event loop). That should be enough.

-- 
:(){ :|:&};: