[Twisted-Python] how to stop dataReceived while waiting a deferred to fire?

[Andrea Arcangeli]

On Mon, Feb 21, 2005 at 04:33:32PM +0100, Andrea Arcangeli wrote:
> code of int32stringreceiver), so I'll try to implement it following the
> linereceiver example.

Ok here we go, I even found a severe bug in the linereceiver code and I
fixed it as well. The bug in linereceiver happens like this:

	resumeProducing()
		calls self.transport.resumeProducing
		calls self.dataReceived
			calls self.stringReceived
				calls self.stopProducing
					calls self.transport.stopProducing
					runs some sql query again
		calls self.transport.resumeProducing <- BUG, invalidated the above stopProducing

The above bug would for sure screwup two sql queries in a raw (i.e.
the deferred callback executing another sql query). Fix is the below one
liner, it's enough to call dataReceived as the last thing in
resumeProducing (I assume for the lower layer the effect of
self.transport.resumeProducing; self.transport.stopProducing will be a
noop as long as we never schedule in between, i.e. never return in
between)

Please checkin the below into trunk. My testcase now passes perfectly
(I'll leave the debug code live so that if something goes wrong I get an
exception and the client is disconnected). I was too lazy to implement
for netstringreceiver (to me netstring receiver looks mostly a waste of
bandwidth and an unnecessary complexity ;). I guess netstring recevier
is more ngrep friendy, but I can't debug with ngrep anyway since it's
all under ssl with certificate validation here. But at least now the
reentrancy bug is fixed and it should be easy to extend to
nestringreceiver if needed.

Thanks for the help!

Index: Twisted/twisted/protocols/basic.py
===================================================================
--- Twisted/twisted/protocols/basic.py	(revision 13001)
+++ Twisted/twisted/protocols/basic.py	(working copy)
@@ -166,9 +166,23 @@
         """
         return error.ConnectionLost('Line length exceeded')
     
+class PauseProducer(object):
+    paused = False
 
+    def pauseProducing(self):
+        self.paused = True
+        self.transport.pauseProducing()
 
-class LineReceiver(protocol.Protocol):
+    def resumeProducing(self):
+        self.paused = False
+        self.transport.resumeProducing()
+        self.dataReceived('')
+
+    def stopProducing(self):
+        self.paused = True
+        self.transport.stopProducing()
+
+class LineReceiver(protocol.Protocol, PauseProducer):
     """A protocol that receives lines and/or raw data, depending on mode.
     
     In line mode, each line that's received becomes a callback to
@@ -188,7 +202,6 @@
     __buffer = ''
     delimiter = '\r\n'
     MAX_LENGTH = 16384
-    paused = False
     
     def clearLineBuffer(self):
         """Clear buffered data."""
@@ -279,21 +292,8 @@
         """
         return self.transport.loseConnection()
 
-    def pauseProducing(self):
-        self.paused = True
-        self.transport.pauseProducing()
 
-    def resumeProducing(self):
-        self.paused = False
-        self.dataReceived('')
-        self.transport.resumeProducing()
-
-    def stopProducing(self):
-        self.paused = True
-        self.transport.stopProducing()
-
-
-class Int32StringReceiver(protocol.Protocol):
+class Int32StringReceiver(protocol.Protocol, PauseProducer):
     """A receiver for int32-prefixed strings.
 
     An int32 string is a string prefixed by 4 bytes, the 32-bit length of
@@ -314,7 +314,7 @@
         """Convert int32 prefixed strings into calls to stringReceived.
         """
         self.recvd = self.recvd + recd
-        while len(self.recvd) > 3:
+        while len(self.recvd) > 3 and not self.paused:
             length ,= struct.unpack("!i",self.recvd[:4])
             if length > self.MAX_LENGTH:
                 self.transport.loseConnection()
@@ -331,7 +331,7 @@
         self.transport.write(struct.pack("!i",len(data))+data)
 
 
-class Int16StringReceiver(protocol.Protocol):
+class Int16StringReceiver(protocol.Protocol, PauseProducer):
     """A receiver for int16-prefixed strings.
 
     An int16 string is a string prefixed by 2 bytes, the 16-bit length of
@@ -351,7 +351,7 @@
         """Convert int16 prefixed strings into calls to stringReceived.
         """
         self.recvd = self.recvd + recd
-        while len(self.recvd) > 1:
+        while len(self.recvd) > 1 and not self.paused:
             length = (ord(self.recvd[0]) * 256) + ord(self.recvd[1])
             if len(self.recvd) < length+2:
                 break