[Twisted-Python] why deferred.setTimeout is not my favorite API method

[Andrew Bennetts]

On Sun, Apr 18, 2004 at 08:48:28AM +0000, slyphon at twistedmatrix.com wrote:
[...]
> 
> again, there's a difference between stopping a long-running or
> long-waiting operation, which is something that deferreds cannot handle
> at all, and indicating that after a certain amount of time, we don't
> care about the result of that operation and would like to consider it an
> erroneous condition.
> 
[...]

On Sun, Apr 18, 2004 at 04:21:11AM +0000, exarkun at divmod.com wrote:
> 
>   I tend to prefer "foo(x, y, timeout=z)" over "foo(x, y).setTimeout(z)".
>   As both require explicit support from the implementor of "foo" (as you
>   correctly note is the only way .setTimeout() can actually work), it
>   seems unnecessary to special-case this form of failure in Deferreds.  I
>   cast my vote for the former.

I think that timeouts, or cancellations in general, are something that
Deferreds should *support*, even though they cannot provide them entirely
automatically.  I'll try to explain why.

Deferred objects are essentially placeholders for a result that isn't
available yet -- a result that is deferred until some later time.
Inevitably, this means there is some associated operation occuring that is
working towards generating that result (even if this operation is passive --
e.g. waiting for a response from a remote server).

Generally, when the recipient of a Deferred is no longer interested in the
result of that Deferred, there is no longer any need to continue the
underlying operation that would create that result.  It would be good if the
underlying operation was aborted in these situations (when possible), to
save resources.

Recipients of Deferreds have no knowledge of the result-generating operation
that is associated with them, nor should they.  In effect, Deferreds are not
just placeholders for pending results, but also intrinsically linked to the
operation that will eventually fulfill them.

Given that Deferreds are fundamentally linked to operations, not just
results, why not use that relationship to provide a consistent interface for
cancellations (and thus also timeouts)?

Here are the potential drawbacks I can think of to this view of Deferreds:
   - Not all operations are sanely interruptable, e.g. the worker thread
     kicked off by deferToThread.
   - foo(x, y, timeout=z) makes more sense for situations where a protocol
     or something requires that timeouts be set before an operation begins,
     and can't be adjusted once the operation is started.  I'm not aware of
     any such operations, though.

There are possibly other reasons why this view doesn't fit neatly -- if so,
please tell me! :)

The drawbacks I can think of aren't enough to dissuade me from this
viewpoint, though.  They do mean that Deferreds may need to only support
cancellations optionally, rather than always, according to what the creator
of the Deferred declares to be appropriate (i.e. the "allowTimeout" I
proposed for setTimeout, or perhaps "canCancel").  I don't see this as a
showstopper.

If people dislike coupling these things this tightly, perhaps Deferreds
should have an "operation" attribute, where things like a cancel method
would go -- does "deferred.operation.cancel('User clicked cancel')" read ok?
This is probably a good idea -- after all, a one-to-one relationship doesn't
necessarily imply that two things should be represented as one object (e.g.
HTTP Request and Response objects are always coupled that I can think of,
but keeping them seperate still feels right).

The operation object would also be a good place to put other functionality
that maybe only some operations could support, e.g. pausing an operation, or
something.  I'm just thinking out loud here.

The problem with the explict operation object is that it would make it even
harder for creators of Deferreds to fully implement things, although they
could just simply not provide one if they wanted to be lazy, I guess.

Any thoughts?

-Andrew.