[Twisted-Python] Reactor Spinning?

[Moxie Marlinspike]

I've seen it clear up before, but it will normally spin for days.  This
state doesn't really impact performance at all (requests are still
snappy), but it is annoying how much CPU time it eats through before a
restart.

- moxie

-- 
http://www.thoughtcrime.org


Werner Thie wrote:
> Does it spin forever or does it clear up after some time?
> 
> I have occasional 'hickups' with heavy loads with responses taking 
> several ten seconds for no apparent reason, the server then working 
> again like nothing happened.
> 
> Werner
> 
> Moxie Marlinspike wrote:
>> Hey Twisted folks,
>>
>> I'm using the twisted.web framework for a high performance HTTP proxy
>> server that very closely resembles the HTTP proxy server example that
>> comes with the twisted package.
>>
>> Under heavy load, I occasionally run into a problem where the reactor
>> appears to start spinning on an "empty" write.  An strace of the process
>> reveals these system calls:
>>
>> epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 8) = 1
>> write(81, "", 0)                        = 0
>> gettimeofday({1272980323, 567613}, NULL) = 0
>> gettimeofday({1272980323, 567696}, NULL) = 0
>> epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 8) = 1
>> write(81, "", 0)                        = 0
>> gettimeofday({1272980323, 567988}, NULL) = 0
>> gettimeofday({1272980323, 568071}, NULL) = 0
>> epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 7) = 1
>> write(81, "", 0)                        = 0
>> gettimeofday({1272980323, 568501}, NULL) = 0
>> gettimeofday({1272980323, 568613}, NULL) = 0
>> epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 7) = 1
>> write(81, "", 0)                        = 0
>> gettimeofday({1272980323, 568978}, NULL) = 0
>> gettimeofday({1272980323, 569084}, NULL) = 0
>> epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 6) = 1
>> write(81, "", 0)                        = 0
>>
>> ...it seems to be waiting for write availability on a FD, trying to
>> write an empty buffer, seeing that it wrote zero bytes, and so waiting
>> for availability again.
>>
>> I've confirmed that when this occurs, whatever FD it is spinning on does
>> correspond with a network socket connected to the proxy.  I've seen this
>> with both Twisted 8.2 and 9.0 running on Linux using both the epoll and
>> select reactors.
>>
>> If anyone has any thoughts about this problem or has seen it before, I'd
>> appreciate any insight that anyone might have.
>>
>> Thanks,
>>
>> - moxie
>>
> 
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>