[Twisted-Python] Large Transfers

[Andrew Bennetts]

On Sat, May 10, 2003 at 06:01:40PM +0200, Bernhard Herzog wrote:
> 
> The tests, AFAICT, simply check the length (as defined by the len
> builtin) of lists, tuples and strings. So if the sender can't send e.g.
> [0] * 1000000, it could just as well send [[0] * 1000 for i in range(1000)]
> which would consume roughly the same amount of memory on the receiver's
> side but would pass all the SIZE_LIMT tests. Using a large dictionary
> seems to work too.
> 
> A better solution for limiting the size seems to be a limit on the
> number of bytes accepted by the receiver for a single PB call.

You seem to be misunderstanding the difference between PB and Banana.  The
limit is in the Banana layer, so that it can remain simple and fast.  A long
list of small items breaks into small pieces at the Banana level already.  

I expect the Unjellying layer could refuse to reassemble lists and
dictionaries beyond a certain size (although it doesn't appear to do that at
the moment).  This is possibly something worth looking into... a
twisted.spread guru could might be able to comment more usefully here.

-Andrew.