[Twisted-Python] File persistence without blocking the main event loop?

[Arun Thampi]

@Itamar - Thanks for that. Didn't know deferToThread uses a threadpool.
@Jean-Paul - Wow thanks, looks interesting..will definitely take a look.

Thanks again guys.

Arun

On Mon, May 18, 2009 at 9:21 PM, Jean-Paul Calderone <exarkun at divmod.com>wrote:

> On Mon, 18 May 2009 08:31:56 -0400, Itamar Shtull-Trauring <
> itamar at itamarst.org> wrote:
> >On Mon, 2009-05-18 at 19:34 +0800, Arun Thampi wrote:
> >> Hi guys - I'm just beginning to learn about Twisted and was wondering
> >> if there is a way to use the built-in Twisted persistence libraries
> >> such as dirdbm etc. without blocking the main event loop? I read about
> >> an approach which uses deferToThread (comment#3
> >> here: http://code.activestate.com/recipes/413609/) but if you are
> >> persisting quite often, doesn't the creation of threads on every
> >> persist-call make it very expensive?
> >
> >deferToThread uses a threadpool, so it's not *very* slow. It does slow
> >things down a bit, so it's better to do chunks of work rather than doing
> >writes separately. That being said, writing to disk in main eventloop is
> >often fast enough in practice, depending on how strong your latency
> >guarantees need to be.
> >
>
> More importantly, don't use dirdbm.  It's not a reliable data storage
> library.  In fact, you probably shouldn't use anything in
> twisted.persisted.
>
> You might be interested in <http://www.divmod.org/trac/wiki/DivmodAxiom>.
>
> Jean-Paul
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>



-- 
It's better to be a pirate than join the Navy - Steve Jobs
http://arunthampi.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20090518/e868303c/attachment.html>