[Twisted-Python] Receiving large packets with TCP

[Glyph Lefkowitz]

Christopher Armstrong wrote:

> dataReceived will be called with arbitrarily-sized chunks of data. You
> can never assume how much data will be given to you. If you're
> expecting more, buffer it until you receive everything you need.

This phenomenon is called "packet fragmentation" and is well-documented 
in various places.  Google for that phrase and you will turn up quite a 
few references :).

In fact, you will notice there are some unit tests in Twisted that do 
something like this:

     for byte in data:
         myProtocol.dataReceived(byte)

just to test that the protocol is still robust in the face of the 
ultimate (1-byte-at-a-time) fragmentation.  (FWIW, no actually existing 
TCP implementation will never actually fragment packets _this_ far, but 
there is nothing saying that it can't.)