[Twisted-Python] Re: How can ftpclient.py work?

[Andrew Bennetts]

On Sat, Jul 26, 2003 at 10:34:54PM -0500, John Goerzen wrote:
> Thanks for the excellent information, Andrew.  I have a few (really!)
> follow-up questions:
> 
> Andrew Bennetts <andrew-twisted at puzzling.org> writes:
> 
> > In general, though, most protocols would simply be doing something like:
> >
> >     def sendRequest(self, data):
> >         # ...
> >         self.transport.write(self.requestHeader)
> >         self.transport.write(data)
> >         # ...
> >
> > And the transport takes care of making sure the data is sent in the right
> > order.  You're probably thinking "but what if two calls are made to
> > sendRequest at the same time?  Isn't there a race condition?"  The answer is
> 
> Oh sure, you guys just turn my whole approach to things on its side,
> and now you know what I'm thinking?  Well, you're right :-)

:)

> But, wouldn't the above code have blocking problems, especially over
> slow links or for large values of data?  (Say data is a 2MB string and
> you're on a modem... does your UI freeze for 15 minutes?  Or is
> write() another layer above the system call that does some sort of magic?)

transport.write doesn't block.  It simply queues the data to be written,
and then immediately returns (actually, if there's nothing queued, it'll try
a non-blocking write immediately -- see
twisted.internet.abstract.FileDescriptor.write for the gory details...).
So, yes, it's a layer above the system call.  A transport is an ITransport
implementor, not the bare socket object.

That said, for large transfers, using producers/consumers might be a better
idea anyway.  An example of this is twisted.web.static.FileTransfer.

> I guess what I'm thinking here is that switching from multi-threaded
> to poll() or select() isn't a "get out of jail free" card.  And while
> I've written various C programs using select() and have a good idea
> what the potential pitfalls are there, Twisted seems to live up to its
> name just enough that I'm having a bit of trouble isolating them here :-)

It's not "get out of jail free", but it's pretty close ;)

> > I hope I've shed some light for you -- let us know if you need more :)
> 
> Thanks -- all I need now is a small flashlight :-)

If you just look at twisted.flashlight... -- oh, wait, we haven't written
that one yet ;)

-Andrew.