[Twisted-Python] HTTP versions

[Andrew Dalke]

Itamar:
> All backwards compatability has to be in the Request class interfaces -
> whatever stuff a end user might be using.

Okay, much easier.  I wasn't sure if the HTTPChannel was meant to
be derivable.

>> What about using just rfc822.Message?  That's in Python 2.2.
>
> Not good enough - MIME stuff should be stored on disk if its large 
> (i.e.
> file uploads). I did it using cgi.FieldStorage, and it was backwards
> compatible too, but it broke twisted.web.distrib, which *sucked* :(
> distrib needs to be fixed so it gets a stream, and then we can mess 
> with
> Request as much as we want.

The Message is used only for parsing the headers, so file uploads
aren't relevant.  As it is now, the headers are processed, line by line.
Partial lines are stored in-memory as "__header", when a full line,
including continuations, is read, it's passed to an in-memory dict,
which is eventually passed to the Request.  There's no disk involvement
at all, and this is all that I plan to change.

My replacement would write those header lines into a cStringIO.StringIO
then use Message to get the headers out of that StringIO.

So I don't know what you mean about storing MIME stuff on disk.

Now, if there are a large number of headers, there might be a problem,
but then again, there's a problem with the current implementation.

BTW, is there a way to limit the amount of data which can be passed
in a header?  I don't like the potential DoS attack possible either
through dumping 1GB of header to the server, or by the hash collision
attack mentioned recently on c.l.py.  One solution is to do as Apache
does and limit the header to N bytes total.

> A lot of this is for historical reasons - Twisted only depending on
> 1.5.2, and then 2.1. 2.2 support is really new.

Oh, I understand that.  Is there a reason to keep the historical
differences?  I think there should be a migration path to make
Twisted use more of existing Python libraries.

> Request methods and attributes that users are likely to use is what
> counts.

Understood.  I don't think that interface code will change all
that much, except that I would like a 2-ple version ... ahh! I
can just make a "version" and a "version_tuple", rather like
what Python does.  Those are just attributes, after all.

					Andrew