[Twisted-Python] Hep and Twisted (Was: Twisted-webserver hangs when serving big files)

[Glyph Lefkowitz]

On Monday, June 9, 2003, at 01:35 PM, Abe Fettig wrote:

> Agreed.  When I first started moving Hep to Twisted I hadn't yet come 
> to
> terms with the "framework, not a library" philosophy, and I didn't use
> much beyond t.i.protocols.  But now I'm comfortable enough with Twisted
> that I'm willing to make Hep a full-blown Twisted Application, and do
> things the Twisted Way whenever possible.

Great to hear it!

> That said, here are the areas that I know of where Hep doesn't do 
> things
> the Twisted way, and my thoughts on how to fix it.  Comments and
> suggestions are welcome.

I'll try to give you a brief survey.

> * The application: Hep should be a .tap-able twisted plugin.

As Moshe says, this should be super-easy.  Not much more to say here.

> * The web server: The current code is Hep CVS is an excellent example 
> of
> what happens when you write Twisted.web code without understanding how
> Resources work.

Yup.  That seems to be the case.

> Please refrain from commenting on just how bad it is -

Sorry, couldn't resist.

OMG IT IS SO BAD LOLOLOLOL

> I started rewriting the web interface in Woven this weekend, so this
> should be fixed soon :-)

Cool.  Much as it's improved, Woven needs more deployments to help it 
with polish :-)

> * Users and authentication:  I have a vague idea that I should be using
> Perspectives and/or Cred for authenticating users and accessing user
> data.  Currently each server in Hep does the work of authentication and
> setting up user access.  It would be nice to have this centralized, so
> you could (for example) see which users are currently connected to Hep.

Itamar and my aforementioned rewrite will help you with this.  Stall 
this a few weeks, and there should be good documentation on how to do 
it.

> * The SMTP server: Currently going through a rewrite in CVS.  I need to
> set up code to manage the outgoing message queue, and keep track of
> which messages have been delivered to which locations.  Is there code 
> in
> twisted.mail to do this? (Keep in mind some messages are being 
> delivered
> to blogs and other non-SMTP locations)

I need to rewrite twisted.words to accomodate use-cases like this at 
some point; however, it keeps falling off the end of my queue.  It 
certainly wouldn't be a good idea to wait for me to come up with that, 
but in a few months we should re-group and see if we can refactor the 
code that we've both written into some generally useful core 
abstraction.

> I think that just about covers it... Thoughts?

Sounds like a good laundry list.  I would be interested to know where 
and if you hit any snags, even if you get past them on your own - I'd 
love to see a paper or something about this.  It would really be nice 
to show to other application authors who are struggling with the same 
sort of issues.  I even have a tentative title - "Submitting to the 
hive mind for fun and profit - making your project Twisted-friendly 
even though you don't want to".  What do you think? :)