[Twisted-Python] Correct pattern to do SSH forwarding in a GUI

[Luper Rouch]

glyph at divmod.com wrote:
> On 04:19 am, luper.rouch at gmail.com wrote:
>> My first attempt was to run the reactor in the main thread, but the 
>> application was "freezing" when e.g. doing a SELECT with 
>> SQLAlchemy/psycopg2 (I guess because psycopg2 just waits for data and 
>> blocks the twisted reactor). So I put reactor.run() in a thread and it 
>> works, but I'm getting strange issues on some platforms (for example I 
>> can't connect at all on *some* OSX computers, getting a "reactor 
>> stopping" log from twisted without any further error), and I feel I'm 
>> doing the whole thing terribly wrong.
> 
> These errors sound like you are making Twisted API calls from threads 
> other than the reactor thread by accident.  There's no obvious fix 
> except for "don't do that" :).

Yes, I did not pay attention to separate Twisted from the main thread at 
all, apart from putting reactor.run() in another thread.

> 
> My suggestion would be to move the SQLAlchemy/psycopg2 into threads, 
> with e.g. deferToThread, rather than putting the reactor into a thread. 
> Then the reactor won't freeze while the blocking SQL stuff happens.
>> So my question is, how to do proper SSH port forwarding with conch in 
>> an application that also uses the tunnels not the asynchronous way ? 
>> If running the reactor in a thread is OK, are there particular things 
>> to watch out for ?
> 
> There's no reason that running the reactor in a thread shouldn't work; 
> it's just generally tricky (especially if you're working with an 
> existing codebase) to completely avoid calling Twisted APIs from other 
> threads.
> 

This implies too much modifications in my code, I think I will put the 
whole Twisted stuff in a separate process and interact with it via 
simple IPC, as I don't use that much of Twisted in the GUI code anyway.