t.f.b.Stage(Instruction) : class documentation

Known subclasses: twisted.flow.controller.Block, twisted.flow.pipe.Pipe, twisted.flow.stage.Callback, twisted.flow.stage.Concurrent, twisted.flow.stage.Map, twisted.flow.stage.Merge, twisted.flow.threads.Threaded, twisted.flow.wrap._Deferred, twisted.flow.wrap._Iterable, twisted.flow.wrap._List, twisted.flow.wrap._String

Abstract base defining protocol for iterator/generators in a flow

This is the primary component in the flow system, it is an iterable object which must be passed to a yield statement before each call to next(). Usage:

  iterable = DerivedStage( ... , SpamError, EggsError))
  yield iterable
  for result in iterable:
      # handle good result, or SpamError or EggsError
      yield iterable

Alternatively, when inside a generator, the next() method can be used directly. In this case, if no results are available, StopIteration is raised, and if left uncaught, will nicely end the generator. Of course, unexpected failures are raised. This technique is especially useful when pulling from more than one stage at a time. For example:

    def someGenerator():
        iterable = SomeStage( ... , SpamError, EggsError)
        while True:
            yield iterable
            result = iterable.next()
            # handle good result or SpamError or EggsError

For many generators, the results become available in chunks of rows. While the default value is to get one row at a time, there is a 'chunked' property which allows them to be returned via the next() method as many rows rather than row by row. For example:

    iterable = DerivedStage(...)
    iterable.chunked = True
    for results in iterable:
        for result in results:
             # handle good result
        yield iterable

For those wishing more control at the cost of a painful experience, the following member variables can be used to great effect:

   - results: This is a list of results produced by the generator, they
              can be fetched one by one using next() or in a group
              together.  If no results were produced, then this is an
              empty list.  These results should be removed from the list
              after they are read; or, after reading all of the results
              set to an empty list

   - stop: This is true if the underlying generator has finished execution
           (raised a StopIteration or returned).  Note that several
           results may exist, and stop may be true.

   - failure: If the generator produced an exception, then it is wrapped
              as a Failure object and put here.  Note that several results
              may have been produced before the failure.  To ensure that
              the failure isn't accidently reported twice, it is
              adviseable to set stop to True.

The order in which these member variables is used is *critical* for proper adherance to the flow protocol. First, all successful results should be handled. Second, the iterable should be checked to see if it is finished. Third, a failure should be checked; while handling a failure, either the loop should be exited, or the iterable's stop member should be set. For example:

    iterable = SomeStage(...)
    while True:
        yield iterable
        if iterable.results:
            for result in iterable.results:
                # handle good result
            iterable.results = []
        if iterable.stop:
            break
        if iterable.failure:
            iterable.stop = True
            # handle iterable.failure
            break

Method	__init__	Undocumented
Method	__iter__	Undocumented
Method	next	return current result
Method	_yield	executed during a yield statement by previous stage