[Twisted-web] rest webservice and big data.

[L. Daniel Burr]

Hi Sébastien,

On Mon, 20 Aug 2007 12:11:33 -0500, Sébastien HEITZMANN <2le at 2le.net>  
wrote:

> Hi
>
> I'm new to twisted programming and I'm wonder how to do the following  
> thing.
>
> I would like to save in a file the content of a PUT method. But i need
> this in a stream mode ( the data may be handred of MB )
>
> Here is a part of my code.
>
> class DataResource(resource.Resource):
>     def __init__(self, dbConnection):
>         resource.Resource.__init__(self)
>
>     def render_PUT(self, request):
>         request.content.seek(0)
>         file('data.dat','wb').write(request.content.read())
>         request.write('OK')
>         request.finish()
>
>
> Thats inspired by an exemple of the oreilly book.
>
> Is there a way to get a coolback juste after the header was sended and
> to handle the reading of the remaining data myself ?
>
> I use only web, not the new web2 api.
>

You cannot stream large files using twisted.web unless you write your
own mechanism.  On the other hand, web2 *does* support streaming file
uploads, so I would advise you to think about using web2 instead, if
you really want streaming.

Someone with deeper knowledge of twisted.web may be able to propose
a strategy for implementing streaming file uploads, but I expect it
would be a fair amount of work, and end up looking similar to what
is already in web2.

> Thanks in advance for any help or link.
>
> Sebastien.
>

Hope this helps,

-- 
L. Daniel Burr