[Twisted-Python] Epoll Memory Issues

[Matthew Glubb]

Hi All,

First of all, I'd like to offer a belated thanks for Twisted and  
congratulations on releasing version 8.

I posted about a year ago about memory issues I was having with  
Twisted and the epoll reactor. I have recently put a simple proxy  
server based on Twisted in to production and have been running a live  
test on a single host for between 300 and 400 concurrent users. The  
server gradually consumes memory to a point where I have to start  
thinking about the experience of my users (the server starts  
swapping) and restart the process. I have been watching this  
behaviour for the last 10 days and in the meantime have been going  
through my code with a fine tooth comb, looking for and testing for  
possible memory leaks. I found a few minor ones, but the memory leaks  
continue.

Today, out of desperation, I switched the reactor from epoll to  
select. Guess what? No more memory leak.

My environment is Linux 2.6, Python 2.5.2, Twisted 8.0.1, twistd. I  
was wondering whether anyone could cast any light on this matter? I  
am pretty sure that this isn't an issue specific to my code but I  
would be interested to hear if anyone else has experienced similar  
problems. I would like to be able to switch back to epoll for obvious  
reasons, but at the moment, I am not confident in running the Twisted  
epoll reactor.

Regards,


Matt

	

Matthew Glubb
Technical Partner

email: matthew.glubb at madebykite.com
phone: 44 (0) 7715 754017
skype: mglubb

Kite
http://madebykite.com

--
GPG: 96FF DE0E 0B7B 37F0 7F8D C54C E285 3D8F 5625 9244