[Twisted-Python] Wrapping a Perspective Broker remote object with a blocking API

[Brian Granger]

> >This is a usage case that we run into regularly.  We have a lot of PB
> >based servers, but sometimes people want to use a client to them from
> >an interactive Python or IPython prompt.  At that point you have to
> >run the reactor in a different thread.
>
> Not true, in fact.  See twisted/conch/stdio.py, for example.  I don't know
> what the Twisted/IPython integration code ended up looking like (or if it
> was ever taken to a satisfactory "completion" point) but I don't think it
> is necessary to have multiple threads for that use case, either.

I have, but this doesn't solve the problem as I recall.  First, these
users want to be able to use PB clients from vanilla python
interpreters and IPython out of the box - not from within some other
process doing the tricks that are in conch/stdio.py.  Am I not correct
that threads are needed in this case?  Second, what this module does
is actually make the terminal asynchronous, which is exactly the
opposite of what these users want.  They *want* a synchronous terminal
and a blocking PB clients.  I hope I am not coming across as angry
about this - I'm not.  To me the lesson is that as long as everything
is really asynchronous anything is possible within the constraints of
this asynchronous universe.  It's the multiple universe things that
makes life complicated (asyn + syn).

> >But, I have never seen a
> >method of putting a blocking API on top of such a client that was
> >_really_ thread safe.  I have seen a few approaches that seem to work,
> >but that are clearly non-thread safe.  I would love to see a robust
> >solution for this problem though - one that could be fully trusted.
>
> Why isn't a solution based on Twisted's thread-safe event-posting API
> (reactor.callFromThread) and the thread-safe event-posting API of whatever
> other thing is being integrated with "really" thread-safe?  Sure, you can
> write programs which will deadlock as a result of application-level bugs
> where one thread is blocked on a request and the other thread needs some
> information from it before the request can be satisfied, but this is inherent
> to wanting to drive an asynchronous API synchronously.  Aside from it, I
> don't think there are any problems with what I described above.

It probably is, but I have never seen an implementation that actually
does this.  Do you know of one?  The ones that I have seen - even
those which use callFromThread have not filled me with thoughts of
robustness, stability and thread safety.  I should look at this again
though.

Also, I should mention that in IPython this is not a show stopper for
us, but it does mean that we currently don't use PB in all of the
places we might otherwise.

Brian



> Jean-Paul
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>