[Twisted-web] Problems uploading data in web2

[James Y Knight]

On Feb 6, 2006, at 5:52 AM, Henrik Thostrup Jensen wrote:

> Hi
>
> I'm trying to use web2 to stream uploading of data, but I ran into  
> some problems. First I tried building from the example in  http:// 
> twistedmatrix.com/projects/web2/documentation/examples/demo.html,  
> however args and files objects in the request did not show  
> anything.  However the length of stream was correct, and using  
> tcpdump, showed the correct value. So I tried building my own  
> uploader which look like this:

What are you actually trying to do? There's two different kinds of  
"streaming upload". The simpler is just a raw data stream. If you  
just want raw data, forget POST and associated form processing, and  
just use request.stream.read() in with a PUT method. That will work  
with curl -T.

However, if you want it to be the result of a form post from a web  
browser, you'll need the MIME multipart support. The default POST  
support (with PostableResource) parses the multipart stream, and  
places all the results in memory or temporary files (depending on  
size), in the args & files dicts. It parses the data incrementally,  
but doesn't give control to your "render" method until all the data  
has been uploaded. If all you're looking for from a streaming upload  
is to not buffer everything in memory, this will do it for you.

If that's not good enough for your application, if you really need  
access to the data as it's being uploaded, you'll need to use the  
lower level machinery in fileupload: the MultipartMimeStream class.  
You give this a raw data stream (request.stream), and the multipart  
boundary from the content-type, then call its read() method  
repeatedly to get back tuples of (fieldname, filename, ctype,  
dataStream). Then you use dataStream.read() to get back data blocks  
and can do whatever you like with them.

I can help explain any of the alternatives in more detail if you let  
me know which you're really interested in.

James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-web/attachments/20060206/3165b528/attachment.htm