[Twisted-Python] Epoll Memory Issues

[Alec Matusis]

I suspect it might be your code. On Twisted 2.5 (with patched epoll bug http://twistedmatrix.com/trac/ticket/2809 ) we have reactors that run for 3-4 months without restart, that do not consume more than 150MB RSS with 8000+ concurrent connections.

----- Original Message ----
From: Matthew Glubb <matt at madebykite.com>
To: Twisted general discussion <twisted-python at twistedmatrix.com>
Sent: Saturday, April 12, 2008 1:53:32 PM
Subject: [Twisted-Python] Epoll Memory Issues

Hi All,

First of all, I'd like to offer a belated thanks for Twisted and  
congratulations on releasing version 8.

I posted about a year ago about memory issues I was having with  
Twisted and the epoll reactor. I have recently put a simple proxy  
server based on Twisted in to production and have been running a live  
test on a single host for between 300 and 400 concurrent users. The  
server gradually consumes memory to a point where I have to start  
thinking about the experience of my users (the server starts  
swapping) and restart the process. I have been watching this  
behaviour for the last 10 days and in the meantime have been going  
through my code with a fine tooth comb, looking for and testing for  
possible memory leaks. I found a few minor ones, but the memory leaks  
continue.

Today, out of desperation, I switched the reactor from epoll to  
select. Guess what? No more memory leak.

My environment is Linux 2.6, Python 2.5.2, Twisted 8.0.1, twistd. I  
was wondering whether anyone could cast any light on this matter? I  
am pretty sure that this isn't an issue specific to my code but I  
would be interested to hear if anyone else has experienced similar  
problems. I would like to be able to switch back to epoll for obvious  
reasons, but at the moment, I am not confident in running the Twisted  
epoll reactor.

Regards,


Matt

    

Matthew Glubb
Technical Partner

email: matthew.glubb at madebykite.com
phone: 44 (0) 7715 754017
skype: mglubb

Kite
http://madebykite.com

--
GPG: 96FF DE0E 0B7B 37F0 7F8D C54C E285 3D8F 5625 9244


_______________________________________________
Twisted-Python mailing list
Twisted-Python at twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20080412/2079e7c3/attachment.htm