[Twisted-Python] Re: Persistence in browsers?

[Tibi Dondera]

Thank you for your answers to my questions, I've been away for a while
and was unable to respond.

I will have no problem with the load on the server, because actually,
each client will be informed of a different event (or very few clients
will follow the same events), and not all events happen at the same time
(but the requirement that the system support lots of clients, and that
for every client, the response time must not be over 5 sec).

I will dig deeper into the twisted-web architecture, to see how "sending
to sleep" HTTP clients is supported.

Thanks again.
_________________
Tiberiu DONDERA


-----Original Message-----
From: twisted-python-bounces at twistedmatrix.com
[mailto:twisted-python-bounces at twistedmatrix.com] On Behalf Of David
Bolen
Sent: Monday, 07 February 2005 18:31
To: twisted-python at twistedmatrix.com
Subject: [Twisted-Python] Re: Persistence in browsers?


"Tibi Dondera" <incoming at pronet-romania.com> writes:

> Short version:
> I have 10.000 - 100.000 web browsers that are connected to my site, 
> and I need to inform them __real-time__ (a max of 3-5 seconds delay) 
> of an event that happened on the server. Is twisted the right way to 
> go, given the fact that it promises asynchronous event handling ?

Independent of twisted versus other methods (which some other notes have
addressed), I think such a load is going to require that you consider
distributing your notification system.  For example, notifying 100000
clients in 5 seconds is 20000 packets per second being generated which
is a non-trivial load on both a machine (not quite sure you can hit this
from application space on typical machines
even) and its corresponding bandwidth (say 64 byte packets including
protocol overhead would saturate a 10Mbit stream).

At the least you might try tiering your notification system - the code
running on your page would connect to one of a pool of servers (ideally
geographically distributed over the network or else backbone latencies
and bandwidths might become an issue), and those servers would maintain
a single connection back to a central server.  This could be tiered
multiple times for more efficiency or scaleability.

Then, your question really becomes one of what sort of fan-out factor do
you need to optimize notifications.  For example, the central
server(s) would notice the change, but only have to notify the next tier
(perhaps 10s of machines).  Those 10s of machines would either each
notify their own next tier, or directly notify a bunch of leaf machines.

With this sort of structure, the number of individual clients that any
given "node" in the system has to support is far less and the system can
grow incrementally by first permitting the ratio of clients to leaf
nodes to increase, and then adding leaf nodes (or another tier) when
needed to suddenly bring that ratio back down.

-- David


_______________________________________________
Twisted-Python mailing list
Twisted-Python at twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python