[Twisted-Python] win32 reactors
glyph at divmod.com
glyph at divmod.com
Mon Mar 31 19:21:37 MDT 2008
On 31 Mar, 10:15 pm, james at mansionfamily.plus.com wrote:
>glyph at divmod.com wrote:
>>These implementations aren't "broken", so much as "limited".
>Well, if they don't honour what the documentation says, then they seem
>broken
>to me, and the documentation doesn't say you might get an unexpected
>64 connection limit. No matter.
This is actually a problem for every reactor; handling of the connection
limit (and there is a connection limit everywhere, even if it's 65535)
is poorly documented and poorly handled.
Patches which rectify this situation for any reactor, either from the
perspective of docs or code, would of course be appreciated.
>>It still doesn't support SSL, though:
>>
>> http://twistedmatrix.com/trac/ticket/593
>Need something patterned very much like Len's stuff
>here http://www.lenholgate.com/archives/000456.html I guess.
The real solution to this problem is really:
* Add an SSH IProtocol implementation that provides ITransport, rather
than building it into the reactor
* Add an API to ITransport to switch protocols, so that the protocol
doesn't need to support protocol switching
The BIO stuff mentioned on that URL is vaguely correct, but the
interesting stuff is exposing it to Python. Now that JP Calderone has
taken over maintenance of PyOpenSSL, we might actually have a chance to
get that done :).
Once you've got an SSL connection that is getting bytes from and
delivering bytes to a transport, you don't need to care about any of the
win32-specific aspects of that post; you just do the I/O using whatever
platform I/O facility you want and the crypto happens completely
disconnected from this socket. iocpreactor already implements enough of
this to work.
Again, if you want to see this happen: write code which does these
things and contribute it.
>>Your suggestion for handle buckets in the win32eventreactor is one
>>which has been suggested before and sounds good to me. Nobody has
>>bothered to implement it yet :). However, even in its absence, I
>>don't believe we'll be completely dropping win32eventreactor because
>>it is useful for integrating with programs that do win32 drawing, even
>>if iocpreactor has higher performance and is more scalable.
>Well, you need a daemon worker thread to call
>GetQueuedCompletionStatus(Ex)
(... blah blah win32 API stuff that I don't understand ...)
>Why do you need an event reactor to support win32 applications, except
>to call MsgWait... rather than GetQueuedCompletionStatus?
My answer here, if I even have one, would be horribly vague: something
about GDI+, or DirectX, or something. Maybe there's no good reason, I
don't know. This level of specificity is probably best served by the
issue tracker; file a ticket. And, if you're going to file a ticket, be
prepared to actually follow up with an implementation. While we want to
maintain support for Windows, the level of energy for doing really
interesting Windows-specific stuff in Twisted is, in a word, "low".
One thing you might want to know before you file that ticket is that
*if* there's a reason for the way things are now, it's because in the
Twisted idiom we always make sure GUI stuff and network stuff is
happening in the same thread. If one of the approaches that you
mentioned requires a dedicated "network" application thread, then that's
probably why we aren't doing it.
>On a similar sort of topic - is there a reason to have lots of
>implementations for POSIX, rather than use libevent or libev?
There is more than one answer to this question. Maybe someone will be
helpful and turn some of these answers into a FAQ on the wiki:
1) Twisted predates libevent by a few years and libev by many years.
One might instead ask why libevent didn't help us develop a C reactor,
rather than writing a whole new library.
2) There's a libevent reactor in a branch. Contribute patches to deal
with the outstanding review comments:
http://twistedmatrix.com/trac/ticket/1930
3) Twisted runs on many platforms, so it supports the lowest common
denominator; that is not libevent, it is Python's "select" module.
4) Neither libevent nor libev (to my knowledge) integrates with any GUI
event loops.
5) Despite many valid rationalizations for its existence, the code in
Twisted was developed organically over many years. The stuff you'll
find here is the stuff that people thought was interesting and had time
to work on. Strategically standardizing on a single low-level
multiplexing mechanism is not something that is particularly fun or
rewarding, especially when getting rid of the old code removes value for
some users. (Not everyone already has libevent installed.)
More information about the Twisted-Python
mailing list