[Twisted-web] twisted.web : limiting file size during upload (using python)

Tue Jul 16 19:55:50 MDT 2013

Pranav

I believe, assuming that this hasn't been worked on last time I looked at
the code, that this is not currently possible. The code is about half way
there.

The request processing function stores the request on the disk if it is
over a certain size. However, once the request has been uploaded, a
function from Python's cgi module is called to parse the data into
request.params. Not only does this lose some information about file uploads
(which can only be recovered by re-parsing the request), but it also loads
all data into memory.

There is also not currently a way to limit either the size of the request,
or the size of individual files in the request.

Assuming that you need a solution and don't want to work on implementing
this in Twisted, I would suggest using nginx as a reverse proxy in front of
Twisted, with the "Upload" module that can be found on the Addons page on
nginx's wiki. This module will place any uploaded files into files on the
disk, and include regular request parameters to tell your Twisted server
where these files are. It is also capable of putting limits in place on the
size of uploaded files.

Cameron

On 17 July 2013 01:58, Pranav Bhardwaj <pranavb85 at gmail.com> wrote:

> Hey
>
> I have a script to be able to upload logs to a twisted web server.
>
> I have the following requirements:
>
> 1) to be able to limit the size of the upload to 10MB.
> 2)  to NOT store file in memory during the request.
>
> Does twisted.web provide these functionality?
>
> Any inputs on these would be welcome!
>
>
>
> Here is the *server part* os the script:
>
>         try:
>             oStream = open(filename, 'wb')
>             oStream.write(request.args['fname'][0])
>             outputStream.close()
>
>         except:
>             # handle exception
>
>
> a test *client script:*
>
>
> <!DOCTYPE html>
> <html>
> <form action="https://serverUrl"
> enctype="multipart/form-data"
> method="post">
> <div class="row">
> <label for="fileUploadToServer">Select file</label><br />
> fname : <input type="file" name="fname" />
> <input type="submit" value="submit">
> </div>
> </form>
> </html>
>
> _______________________________________________
> Twisted-web mailing list
> Twisted-web at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-web/attachments/20130717/b1660084/attachment.html>