[Twisted-Python] File persistence without blocking the main event loop?

[Itamar Shtull-Trauring]

On Mon, 2009-05-18 at 19:34 +0800, Arun Thampi wrote:
> Hi guys - I'm just beginning to learn about Twisted and was wondering
> if there is a way to use the built-in Twisted persistence libraries
> such as dirdbm etc. without blocking the main event loop? I read about
> an approach which uses deferToThread (comment#3
> here: http://code.activestate.com/recipes/413609/) but if you are
> persisting quite often, doesn't the creation of threads on every
> persist-call make it very expensive?

deferToThread uses a threadpool, so it's not *very* slow. It does slow
things down a bit, so it's better to do chunks of work rather than doing
writes separately. That being said, writing to disk in main eventloop is
often fast enough in practice, depending on how strong your latency
guarantees need to be.