[Twisted-Python] Question about "starving" the reactor

[Croepha]

On Mon, Jun 20, 2011 at 11:23 AM, Luke Marsden <
luke-lists at hybrid-logic.co.uk> wrote:

> On Mon, 2011-06-20 at 11:09 -0400, Glyph Lefkowitz wrote:
> > On Jun 20, 2011, at 7:39 AM, Phil Mayers wrote:
> > > On 20/06/11 11:29, Orestis Markou wrote:
> > >> It might be there's no way to do guarantee something like that, so it
> > >> might be just that everyone should be careful about this.
> > >
> > > I don't think you can stop callers of a function doing silly things
> with
> > > the return value.
> >
> > +1.  The best way to deal with this is to make your APIs nice and
> > simple, and their implementation straightforward.  If you try to do
> > weird tricks to make callbacks on your Deferreds cooperative, then you
> > might break an otherwise reasonable strategy on the part of the client
> > code to be well-behaved on their part.
> >
> > For the code that is itself trying to be well-behaved, there are
> > things like twisted.internet.task.cooperate.
>
> There might not be any way to stop consumers of your Deferreds executing
> blocking operations, but when trying to track down which pieces of code
> are the culprits, I found exarkun's BigTimesliceTimer very useful:
>
> http://twistedmatrix.com/trac/browser/sandbox/exarkun/btt.py


On that note, I made an updated version of the btt:
http://pastebin.com/uBbSuDr6

I know i'm digging up ancient history here, but I thought it was relevant


>
> It uses a reactor-independent timing mechanism (the setitimer sigcall on
> Linux, contact me if you want the FreeBSD version) to set an alarm which
> the reactor then has to "race" to unset, else the itimer handler gets
> pre-emptively executed.  When the alarm runs, it prints a traceback of
> the current execution point in the "client" code.  By setting the alarm
> frequency sufficiently low and by watching for sufficiently long you can
> "sample" the code which is running while the reactor is blocked.  Code
> paths which show up frequently are therefore *more likely* to be the
> culprits.
>
> One day it would be nice to turn this into some kind of statistical tool
> for highlighting which code paths are the "hot-spots" in your code, so
> that you can optimise the "blockiest" bits first.  Premature
> optimisation, etc.
>
> Hope this helps; it helped me :-)
>
> --
> Best Regards,
> Luke Marsden
> CTO, Hybrid Logic Ltd.
>
> Web: http://www.hybrid-cluster.com/
> Hybrid Web Cluster - cloud web hosting
>
> Phone: +447791750420
>
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20110929/24cb5fc1/attachment.html>