[Twisted-Python] How Twisted is This?

[Greg Fortune]

On Thursday 17 July 2003 02:55 am, Tommi Virtanen wrote:
> On Wed, Jul 16, 2003 at 06:39:09PM -0400, Bob Ippolito wrote:
> > watch out for when writing a web spider, for example.  It just doesn't
> > make sense to intentionally design something that behaves like:
> > ______/\___________/\_____ when you really have an extremely
> > parallelizable task at hand.  Checking site A is in no way dependent on
> > checking site B, so there's no reason to intentionally make them happen
> > at the same time if you know better.  We're not talking about a whole
> > lot of code here.  Leveraging the fact that 99% of what needs to be
> > done is glue code between stuff that's already in twisted, a prototype
> > of his application could be easily be done in less than fifty lines
> > using the scalable approach.  In fact, I think that the "wake up and do
> > stuff every N minutes" approach would actually end up being a longer
> > and easier to screw up implementation.
>
> 	You all should probably read up on what the Nagios project
> 	thinks about randomizing monitoring intervals. They have a
> 	stable open source product that can scale reasonably well,
> 	and have good opinions on that subject.

I'm pretty sure no one is talking about randomizing...  One solution is a 
fixed interval check and the other is a "as needed" check.  The only ways I 
can imagine that a "as needed" check would be more costly is if the monitors 
were queued very frequently and the controller ended up waking up/sleeping 
very very short intervals or if a huge number of monitors were queued with 
start times far into the future and the controller took a long time to search 
the list for the next pending monitor.  An intelligent insertion routine 
could solve the second issue easily and if a five minute check is sufficient, 
I can't imagine the first issue being a problem...


btw, I couldn't find that doc with a quick look at http://www.nagios.org/.  Do 
you have a direct link 'cause I'm curious regardless :)

Greg