[Twisted-web] Debugging truncated response.. (REPOST to fix newlines)

[A Desai]

Update: I added a routine dataReceivedBufLen() to the LineReceiver class and call it to check if there is any pending data when the lost connection callback is called on the connection.  There is no pending buffered data when the connection is lost.  This implies I am using twisted properly, and maybe the bug is in the program on the other end of the connection.

Thanks,
Arun

--- On Fri, 6/10/11, A Desai <ardesai at yahoo.com> wrote:

From: A Desai <ardesai at yahoo.com>
Subject: Re: Debugging truncated response.. (REPOST to fix newlines)
To: twisted-web at twistedmatrix.com
Date: Friday, June 10, 2011, 3:49 PM

I am trying to debug a scenario wherein a program has an http channel/transport 
that receives an HTTP response; and in some cases (5 out of 10,000) the 
channel/transport invokes the lost connection callback while the amount of data 
received is less than the content length.  The connection is used as a producer 
to write to another connection.

I am trying to debug to check whether:
(1) the last chunk (20K+) of data sitting in the incoming response is not being 
propagated up, e.g. due to a prior pauseProducing() call, or 

(2) the data was not sent from the peer (e.g. connection closed without using so_linger)

In general, I "presume" lost connection callback on a producer socket would not 
be called if there is pending (paused) data that has yet to be written a 
consumer (socket).  I am also assuming that
 the assert in 
internet/abstract.py::resumeProducing() [self.connected and not self.disconnecting] is valid.  
I.E: If someone can confirm that scenario (1) above is not possible in twisted (unless a bug).

By the time the lost connection callback is called, the transport channel is gone.  
I would have to instrument either tcp.py, basic.py, or abstract.py to track this.

Any clues/suggestions welcome.

Thanks.
-Arun


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-web/attachments/20110610/24c86484/attachment.htm