[Twisted-Python] Bloat (was Re: [Twisted-commits] DependentMultiService - chained start/stop of services in a sensible order)

[Bob Ippolito]

On Sunday, Apr 13, 2003, at 20:32 America/New_York, Glyph Lefkowitz 
wrote:

> This patch strikes me as unnecessary:
>
> On Sunday, April 13, 2003, at 03:48 PM, etrepum CVS wrote:
>
>> Modified files:
>> Twisted/twisted/internet/app.py 1.89 1.90
>>
>> Log message:
>> DependentMultiService - chained start/stop of services in a sensible 
>> order
>
> but I would like to discuss it and other patches like it.  Not to pick 
> on bob here; there isn't really a policy on avoiding bloat in Twisted, 
> so it can't be said that this is a violation of anything in 
> particular.  I just think we need one.
>
> First, my question is: do we have more than one or two users who 
> actually need this functionality?  If it's a real, present need for a 
> wide variety of applications, then much of this criticism does not 
> apply.

Do we have more than two users that _know_ they need/want this 
functionality?  Probably not, yet.

Here's a use case.  Let's say you have some sort of service hierarchy 
that resembles a system startup.  The way Darwin does this is that it 
has plist files bundled with the actual "rc" scripts that determine 
which services get started and in what order by SystemStarter.  Here's 
an example plist file (in NeXT format, because XML is retarded) for the 
LDAP service:
{
   Description     = "LDAP";
   Provides        = ("LDAP");
   Requires        = ("Network Configuration", "DirectoryServices");
   OrderPreference = "None";
}

In this case, in order for LDAP to start, the services that provide 
Network Configuration and DirectoryServices must already be running.  
It doesn't care which ordering bucket it sits in (OrderPreference can 
be Last, First, etc.), and it provides LDAP functionality to other 
services that want it.

DependentMultiService is a very crude basis for such a mechanism, 
because I don't need that kind of functionality yet.  However, it does 
provide a pretty decent base class for a future Twisted "SystemStarter" 
equivalent... which we *should* want to have someday.

Thinking like this could potentially make things such as tying 
authorizers to pb services and whatnot easier, if they're done in a 
service collection manner rather than just tied to the same parent.  
However, as you said, app and cred are already nasty and need a major 
refactoring.

>
>> +        # Warning:  On failure, service stops are not
>> +        # chained.  However, they will stop in the proper order.
>
> "utility" patches like this will invariably fail to take edge cases 
> into account well.  I wouldn't say it's immediately obvious how 
> edge-cases *should* be taken into account, personally, but I would 
> guess that half-changing the semantics of the ordering in case of 
> failure is not the correct thing to do.

Basically, it's not at all obvious what you do if you have a failure in 
a collection of interdependent services.  On startup or shutdown, if 
one fails, you obviously aren't going to be able to say that whole 
collection is started or stopped properly, so I make the remainder of 
the running services shut down immediately.  The only difference in 
semantics is that once a failure happens, the service stopping is no 
longer chained (the rest of the running services get stopService called 
immediately).  It really depends on what kind of services you have to 
determine whether you want to try and chain this or not.. but in most 
cases, startService and stopService don't return deferreds anyways, and 
in even more cases, you don't expect them to fail.  I'll go ahead and 
add an __init__ option that lets you decide which behavior gets chosen 
in the case of a failure.

Also, the fact that I'm returning a deferred here either way means that 
it's probably possible that you could even set a timeout.. which is 
kind of cool, but I've said before that the Deferred timeout API can 
cause some issues (it can't actually make something NOT happen.. which 
is really unfortunate).  I really think there should be an API to 
cancel Deferreds, and the methods that return them should be refactored 
to respect a cancel whenever possible.

-bob