[Twisted-Python] Twisted receiving buffers swamped?

[Tobias Oberstein]

>>I am aware of http://twistedmatrix.com/documents/14.0.0/core/howto/producers.html, but that seems to cover the sending side only.

>It covers the receiving side as well.  If you pauseProducing() on a transport, it stops calling dataReceived on its transport.

Not sure I understand that. But you say, this will stop Twisted reading incoming data from a socket into userspace? And hence TCP backpressure results?

>>What's the cause? What can I do?

>My initial hypothesis is that netperf is sending traffic but not bothering to receive it.

I haven't looked through the netperf sources .. but I guess netperf will send as fast as the receiving side can digest .. only throttle down because of TCP backpressure, not app-level flow-control.

>If this hypothesis is correct, then self.transport.registerProducer(self.transport) should solve the problem.  Presuming that there is no problem with crossing the streams - I don't think i've ever done that particular incantation, >and I'm almost shocked it's taken this long to come up :).

Unfortunately, it doesn't seem to work (the problem persists):

https://github.com/oberstet/scratchbox/blob/master/python/asyncio/tcp_echo_server_tx.py

http://picpaste.com/pics/Clipboard07-HlqQmTW0.1420188656.png

Btw: the problem also arises when running over real network .. at least fast networks. I tested on fully switched 10GbE.

And: asyncio (Trollius that is), has the same issue. The target server just gets swamped .. and then killed.

Cheers,
/Tobias