[Twisted-Python] Wrapping a Perspective Broker remote object with a blocking API

[Brian Granger]

I take most of this back.  I just saw that in twisted.internet.threads
there is now a

blockingCallFromThread

function that does this exactly.  Last time I looked at the ticket
that led to this, there were still some things being worked out.  This
look very nice though!

Brian

On 9/4/07, Brian Granger <ellisonbg.net at gmail.com> wrote:
> > >This is a usage case that we run into regularly.  We have a lot of PB
> > >based servers, but sometimes people want to use a client to them from
> > >an interactive Python or IPython prompt.  At that point you have to
> > >run the reactor in a different thread.
> >
> > Not true, in fact.  See twisted/conch/stdio.py, for example.  I don't know
> > what the Twisted/IPython integration code ended up looking like (or if it
> > was ever taken to a satisfactory "completion" point) but I don't think it
> > is necessary to have multiple threads for that use case, either.
>
> I have, but this doesn't solve the problem as I recall.  First, these
> users want to be able to use PB clients from vanilla python
> interpreters and IPython out of the box - not from within some other
> process doing the tricks that are in conch/stdio.py.  Am I not correct
> that threads are needed in this case?  Second, what this module does
> is actually make the terminal asynchronous, which is exactly the
> opposite of what these users want.  They *want* a synchronous terminal
> and a blocking PB clients.  I hope I am not coming across as angry
> about this - I'm not.  To me the lesson is that as long as everything
> is really asynchronous anything is possible within the constraints of
> this asynchronous universe.  It's the multiple universe things that
> makes life complicated (asyn + syn).
>
> > >But, I have never seen a
> > >method of putting a blocking API on top of such a client that was
> > >_really_ thread safe.  I have seen a few approaches that seem to work,
> > >but that are clearly non-thread safe.  I would love to see a robust
> > >solution for this problem though - one that could be fully trusted.
> >
> > Why isn't a solution based on Twisted's thread-safe event-posting API
> > (reactor.callFromThread) and the thread-safe event-posting API of whatever
> > other thing is being integrated with "really" thread-safe?  Sure, you can
> > write programs which will deadlock as a result of application-level bugs
> > where one thread is blocked on a request and the other thread needs some
> > information from it before the request can be satisfied, but this is inherent
> > to wanting to drive an asynchronous API synchronously.  Aside from it, I
> > don't think there are any problems with what I described above.
>
> It probably is, but I have never seen an implementation that actually
> does this.  Do you know of one?  The ones that I have seen - even
> those which use callFromThread have not filled me with thoughts of
> robustness, stability and thread safety.  I should look at this again
> though.
>
> Also, I should mention that in IPython this is not a show stopper for
> us, but it does mean that we currently don't use PB in all of the
> places we might otherwise.
>
> Brian
>
>
>
> > Jean-Paul
> >
> > _______________________________________________
> > Twisted-Python mailing list
> > Twisted-Python at twistedmatrix.com
> > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> >
>