[Twisted-web] web client & "TimeoutError: User timeout caused connection failure"

[Lenny G Arbage]

I'm running into a problem under high client (not
server) load.  If I create a client process that makes
only light requests to the server, everything works
perfectly.  If, however, I spawn a bunch of client
requests simultaneously, using HTTPDownloader and
HTTPClientFactory (or getPage and downloadPage, it
doesn't seem to matter), I invariably get a bunch of
"TimeoutError: User timeout caused connection failure"
failures.  These failures have no stack associated
with them, so it isn't clear where they are
originating, but it appears that they are due to
timeouts on waiting for the connection setup, not on
the response.

I know that there is a lot of activity on redoing the
client in web2, but I didn't see any mention of this
particular problem (or potential ways to avoid it) in
the list archives.  Is this a known problem?  Is there
a workaround I can rely on until something better
comes along in web2?

The breakpoint (on my 2.8Ghz Pentium) seems to be
around 40-50 simultaneous client connections.  On my
slower 2.53 Ghz machine, it is closer to 70-80.

thanks,
Lenny

(I can produce a short example that reproduces the
behavior if needed)

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com