[Twisted-Python] Reactors and New Asynchronous APIs

[Justin Johnson]

On 5/5/05, Patrick Mézard <pmezard at gmail.com> wrote: 
> 
> Hello,
> I have several questions about twisted reactors and extending the twisted
> framework with new asynchronous APIs.
> 
> 1- What is the status of the IOCP reactor? Is it stable? If it is, is 
> there any
> reason for not selecting it as the default reactor for win32 NT platforms?

 The IOCP reactor doesn't support spawnProcess, among other things, which is 
why I currently can't use it. The author (PenguinOf in IRC) was also going 
to submit a list of bugs he knows about. Here are the notes I jotted notes 
down in IRC.

   1. always resets connection instead of closing it nicely 
   2. need half-close support 
   3. need fancy optimizations, like multiple threads and multiple 
   outstanding read requests per socket 
   4. redesign proactor 
   5. reactor.shutdown doesn't cancel


2- I am considering the integration of a custom database API (like Berkeley 
> DB)
> in a twisted application. I could easily wrap the API in a C++ python 
> module and
> access it synchronously from twisted, at least in a prototyping phase.
> Obviously, if the blocking calls are too long it would make the whole 
> framework
> completely useless. Then, the calls could be made from twisted managed 
> threads.
> Assuming the module code was written to minimize GIL contention it would
> probably greatly improve the situation. The most efficient solution would 
> be
> to plug the asynchronous API directly into the reactor:
> 
> Is it possible to extend reactors with new asynchronous API implemented in
> separate client modules?
> 
> With my little knowledge of the twisted framework and quick glances at the
> selectreactor and iocpreactor implementation, I think the answer is no.
> The practical solution would be to duplicate the reactor code and extend 
> it
> with the new API calls.
> 
> I do not know if it would be easy or not to provide a pluggable 
> asynchronous
> framework *at C module level*. Basically, the reactor is nothing else than 
> a
> thread-safe event queue with a (post/get)-event API. If this abstraction 
> can be
> implemented at C level, then plugging new asynchronous handlers would be 
> clearly
> doable, _iocp.c is a good example for that. Now, I am not really 
> knowledgeable
> (yet) about python C module coding, and I guess it is not exactly easy to 
> bind
> C callbacks from one python C module into another using normal python code 
> as
> glue. In the end, this is just another plugin/component system like COM to
> write so...
> 
> Any thoughts?
> 
> Patrick Mézard
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20050505/5dc570e5/attachment.htm