[Twisted-Python] Looking for some advice debugging issue with transport.write calls

[Glyph Lefkowitz]

> On Apr 28, 2015, at 10:55 AM, Brian Costlow <brian.costlow at gmail.com> wrote:
> 
> Okay, figured this out.

Glad to hear it!

> Abstract of the issue: I am a dumbass.

We all make mistakes.  And a lot of people make this specific one :-).

> First, Itamar gets to thrash me soundly,

No need for violence!

> as there was a bug in some code (not shown in my example) that is not properly tested. That code was responsible for "turning off" the protocol instance if  connectionLost method was called, by doing some cleanup then redefining check_for_send in the instance as a no-op to stop it from pushing itself back on the reactor loop.

Ain't it always the way.

We always say "please simplify the test case" and then the asker always says "but it only happens when it's super complex".  The act of simplifying the test case itself often pinpoints the problem (as it appears to have done in your case).  Honestly this was a pretty quick turnaround and I appreciate the 

> Question: I'm assuming there's a good reason transport.write is written so it doesn't error and fails silently even though its underlying connection is not connected anymore. As part of grokking the guts of this thing I've been using for a decade...I'm curious to know why.

The main reason is "predictability".

First, consider this function:

def sendSomeCommand(self):
    self.transport.write(b"DO-SOMETHING\r\n")
    self.transport.write(b"DO-SOMETHING-ELSE\r\n")

Simple enough, right?  Sends two commands?

We could easily find out between the first call to write() and the second one that the connection has dropped.  Twisted optimistically invokes the send() syscall when it can (if the write buffer is empty when write() is called).  This means that if the buffer happens to be empty when this method is called and if the underlying OS already knows that the connection is closed, we might be able to invoke connectionLost between the first and second write() invocation.  Since you can never really know whether those invocations are happening in isolation or together, this apparently simple function needs to turn into

def sendSomeCommand(self):
    if not self.transport.disconnected:
        self.transport.write(b"DO-SOMETHING\r\n")
    if not self.transport.disconnected:
        self.transport.write(b"DO-SOMETHING-ELSE\r\n")

Also, if we were to aggressively report errors like this, users of Twisted might get the impression that a successful return from write() means that the bytes were actually sent.  What does "actually sent" mean?  This is a surprisingly complicated question with a correspondingly complicated answer.  Sent to the ethernet card? To the local network? Acknowledged by the router on the other side? By the kernel of the OS on the other side? By the application container on the other side? And so on and so forth.  Since you already have no idea exactly where in your stream of write() calls the other side's idea of the stream has terminated unless you have application-level acknowledgement, forcing users to handle exceptions for the case where it definitely wasn't received, while still not giving them any indication that it might not have been received, is a misleading and inconvenient API.

So if you want to stop sending data to a transport that you have been notified about in connectionLost, just stop calling write() :).

Hopefully this sheds some light onto transport.write's design.

-glyph
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20150428/bfacfb70/attachment-0002.html>