[Twisted-Python] Blocking on deferreds during unit tests (e.g. deferredResult)

[Jp Calderone]

On Sat, 7 May 2005 11:02:09 -0400, Kevin Dangoor <dangoor at gmail.com> wrote:
>On 5/6/05, Christopher Armstrong <radeex at gmail.com> wrote:
>> I'd suggest 1) not writing a unit test framework, and instead using
>> trial, and 2) instead of using stackless, use greenlet + my
>> gthreadless module that integrates greenlets and Deferreds
>> (sandbox/radix/gthreadless.py).
>
>gthreadless looks quite cool! From things I've seen along the way, it
>seems that many people are turned off by the callback style of
>programming required by an async framework like Twisted. Using
>gthreadless, your code looks just like normal synchronous code.
>
>I'm only a little familiar with greenlet. Is there any significant
>disadvantage to using this approach extensively? For example, is the
>overhead very large?

  The main advantage is also the main disadvantage: when using gthreadless, rather than function callbacks, it becomes less obvious where context switches occur, leading to an increased likelihood of introducing concurrency-related bugs (primarily due to switches out of non-atomic operations).

  Unlike pre-emptive threads, such programming errors can be avoided entirely without the use of locks, but doing so may require more skill and attention to detail than would be required when writing the same code without gthreadless.

  This is somewhat unfortunate, since programmers least familiar with the concurrency problems solved by Twisted's current mechanisms -- function callbacks -- seem to be the most likely to be attracted by solutions like gthreadless while at the same time being the least prepared to develop software correctly with them.

  Of course, this isn't to say gthreadless isn't useful or an improvement over explicit use of Deferreds, just that caution needs to be taken in its use.  gthreadless documentation to the effect of the above paragraphs might go some ways towards letting people use it correctly, too (though since the module is still only in Chris' sandbox, I don't expect him to go out and write this right away; conversely, if someone were to contribute documentation, this might help gthreadless get released, either as part of Twisted or in a stand-alone capacity).

  Jp