[Twisted-Python] Hep and Twisted (Was: Twisted-webserver hangs when serving big files)

[Abe Fettig]

On Sun, 2003-06-08 at 00:26, Glyph Lefkowitz wrote:
> On Saturday, June 7, 2003, at 09:25 PM, Abe Fettig wrote:
> 
> > [look of horror on my face]
> 
> You and me both :)
> 
> > The web code in Hep 0.3.2 was crudely ported to Twisted from existing
> > asyncore-based code.  It should _not_ be viewed as an example of good
> > Twisted practices.  And, for that matter, neither should the web code 
> > in
> > Hep CVS :-).
> 
> We're always around, here and on IRC, if you need help to get things 
> better in-sync with Twisted "best practices".
> 
> Hep is a pretty interesting project, so it will likely attract this 
> kind of attention in the future as well.  We both have an interest in 
> seeing it more in-sync with the overall Twisted vibe, I think.

Agreed.  When I first started moving Hep to Twisted I hadn't yet come to
terms with the "framework, not a library" philosophy, and I didn't use
much beyond t.i.protocols.  But now I'm comfortable enough with Twisted
that I'm willing to make Hep a full-blown Twisted Application, and do
things the Twisted Way whenever possible.

That said, here are the areas that I know of where Hep doesn't do things
the Twisted way, and my thoughts on how to fix it.  Comments and
suggestions are welcome.

* The application: Hep should be a .tap-able twisted plugin.  At the
moment it's not.  This is assuming that it's possible to create a plugin
that combines multiple kinds of servers (http, pop3, imap, etc) into a
single application.  Also I believe that if I was using a twisted
Application object there would be some kind of official Twisted way to
share objects between my different servers.  It's been a while since I
read the plugin docs.  I'll do so soon.

* The web server: The current code is Hep CVS is an excellent example of
what happens when you write Twisted.web code without understanding how
Resources work.  Please refrain from commenting on just how bad it is - 
I started rewriting the web interface in Woven this weekend, so this
should be fixed soon :-)

* Users and authentication:  I have a vague idea that I should be using
Perspectives and/or Cred for authenticating users and accessing user
data.  Currently each server in Hep does the work of authentication and
setting up user access.  It would be nice to have this centralized, so
you could (for example) see which users are currently connected to Hep.

* The SMTP server: Currently going through a rewrite in CVS.  I need to
set up code to manage the outgoing message queue, and keep track of
which messages have been delivered to which locations.  Is there code in
twisted.mail to do this? (Keep in mind some messages are being delivered
to blogs and other non-SMTP locations)

I think that just about covers it... Thoughts?

Abe