[Twisted-Python] custom FTP server is blocking

[Paul Wiseman]

On 20 March 2015 at 12:19, Adi Roiban <adi at roiban.ro> wrote:
> On 20 March 2015 at 11:16, Paul Wiseman <poalman at gmail.com> wrote:
>> On 19 March 2015 at 20:38, Louis D. Burr <ldanielburr at me.com> wrote:
>>> Hi Paul,
>>>
>>>> On Mar 19, 2015, at 1:18 PM, Paul Wiseman <poalman at gmail.com> wrote:
>>>>
>> I want to stream the http request back to the client connected over
>> FTP. I guess I could do that with io.BytesIO right? I can't figure out
>> how to wire it up though. I've added agent.request which is firing
>> back a twisted.web.client.Response but I'm not sure how to stream the
>> data from that into the BytesIO object so that _FileReader has a file
>> like interface to read from.
>
> Without an example code use on your side I find it hard to help here.
> Do you have an http://sscce.org/ for your problem?
>
> Most probably you will need to implemented your own FileReader which
> will hook into the response.deliverBody with a protocol which will
> forward the data from dataReceived to your FTP data connection...all
> this using a consumer/producer API.
>

I just wrote an example that shows what I'm trying to do, it runs an
anonymous ftp server on 2121 of the local file system and any file you
try to download show should stream the body of http://www.yahoo.com
instead of the file contents.

There's a couple of reasons I think this doesn't work, I think I need
to return the _FileReader right away so it can begin streaming, but I
don't know how to make it wait for the deferred which is created in
the streamBody function, as this will file once the streaming is
complete.

I also think another problem is that as the stream is written the file
pointer on the BytesIO object will be at the end of the file, so I
guess the reader would need to seek to the right place, then tell to
the end afterwards so the writer continues to stream properly?

I'm not sure how to hook this up!

import io
from twisted.python import log
from twisted.internet import reactor
from twisted.internet.defer import Deferred
from twisted.internet.protocol import Protocol
from twisted.web.client import Agent, ContentDecoderAgent, GzipDecoder
from twisted.protocols.ftp import FTPFactory, FTPRealm,
FTPAnonymousShell, _FileReader
from twisted.cred.portal import Portal
from twisted.cred.checkers import AllowAnonymousAccess, FilePasswordDB
from twisted.internet import defer

agent = ContentDecoderAgent(Agent(reactor), [('gzip', GzipDecoder)])

class StreamWriter(Protocol):
    def __init__(self, finished, stream):
        self.finished = finished
        self.stream = stream

    def dataReceived(self, bytes):
        self.stream.write(bytes)

    def connectionLost(self, reason):
        print 'Finished receiving body:', reason.type, reason.value
        self.finished.callback(None)


def streamBody(response, stream):
    finished = Deferred()
    response.deliverBody(StreamWriter(finished, stream))
    return finished

def openForReading(self, path):
    d = agent.request("GET", "http://www.yahoo.com")

    stream = io.BytesIO()
    d.addCallback(lambda resp: streamBody(resp, stream))
    d.addErrback(log.err)

    return defer.succeed(_FileReader(stream))

def main():

    FTPAnonymousShell.openForReading = openForReading

    p = Portal(FTPRealm('./'),
           [AllowAnonymousAccess(), FilePasswordDB("pass.dat")])

    f = FTPFactory(p)

    reactor.listenTCP(2121, f)
    reactor.run()

if __name__ == "__main__":
    main()


> Cheers
> --
> Adi Roiban
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python