[Twisted-Python] most efficient new connection rate limiting?

Alec Matusis matusis at yahoo.com
Wed Feb 4 17:37:04 MST 2009


We need to limit the new connection rate from a given IP, such that for
example 10.10.1.10 cannot connect more often than 10 times per minute.

This is a high-volume TCP Twisted server, with about 500 new distinct IP
connections per second in the normal state (and we run 8 of these on each 8
core server, so ~4000 new conns/sec per box).

I am trying to find the least CPU intensive approach for this.

1) Create an dictionary {ip1:count1, ip2: count2, .} in the server, and
check the counts for each incoming connection. 
Disconnect with transport.loseConnection() if the threshold for ip:count is
exceeded.
Reset this dictionary to empty dict {} every minute with reactor.callLater
timer.

2) Use some Twisted rate limiter API that I am not familiar with?

3) Use iptables rate-limiting module like so:
iptables -I INPUT -p tcp --dport 8888 -i eth0 -m state --state NEW -m recent
--set
iptables -I INPUT -p tcp --dport 8888 -i eth0 -m state --state NEW -m recent
--update --seconds 60 --hitcount 10 -j DROP

Which one of these approaches would you recommend?





More information about the Twisted-Python mailing list