[Twisted-Python] epoll reactor problems

Thomas Hervé therve at free.fr
Wed Apr 11 02:24:58 MDT 2007


Quoting Alec Matusis <matusis at matusis.com>:

> We just switched 2 types of production servers to epoll reactor( Twisted
> 2.5) from poll reactor (Twisted 2.2).
>
> The CPU%% utilization of the first type of server that does not do much
> except occasionally pushing messages to about 5000 clients dropped from
> about 40% to 8%, which is very good.

That's great.

> The second type of server is more complicated. The CPU utilization of that
> server (as measured by top) went down from 40% to about 15% after switching
> to epoll.
>
> Here is the problem: after about 10min of running that server with CPU%%
> staying at about 15%, the CPU suddenly jumps to 99.9% for that process and
> just stays there. We reproduced this several times. The server remains
> responsive, even when top shows 99.9% CPU. (Which is very different from
> 99.9% CPU from real load when we used poll- we are intimately familiar with
> the performance in that regime unfortunately.)
>
> The kernel is 2.6.11.4-21.12-smp
>
> Can anybody help with this 99.9% CPU epoll problem?

We would probably need more information. What's your version of python? 
Can you
provide a reproductible example? Did you try to do a strace on your running
server to see what's going on?

As your load was already 99 before using epoll, can it be an application
problem?

-- 
Thomas






More information about the Twisted-Python mailing list