[Twisted-Python] Re: Re: Re: Re: rewrite of flow.py completed

[Philippe Lafoucrière]

Clark C. Evans wrote:

> On Wed, Apr 16, 2003 at 02:03:06PM +0200, Philippe Lafoucrière wrote:
> | I solved the problem. It was a missing import (QueryIterator). But any
> | exception is raised when using flow. It's too hard to debug, even for
> | few lines of code.
> 
> Yes, I know... better exception handling is next on me list.
> 
> ;) Clark

Thank you for the documentation Clark. Here's a little doc patch that just
add some titles and a new part on QueryIterator (which is not good at all
:( )

Index: flow.html
===================================================================
RCS file: /cvs/Twisted/sandbox/flow.html,v
retrieving revision 1.3
diff -u -r1.3 flow.html
--- flow.html   17 Apr 2003 08:00:11 -0000      1.3
+++ flow.html   17 Apr 2003 10:51:09 -0000
@@ -63,10 +63,10 @@
 than blocking, the entire state of the iterator chain must be saved so that
 it can be resumed later.   This is what the flow module does.</p>

-<h3>Iterators and Wraps</h3>
+<h3>Iterators and generators</h3>

 <p>An iterator is basically an object which produces a sequence of values.
-Python's iterators are simply objects with an <code>__iter__()</code>
+Python's iterators are simply objects with an <code>__iter__()</code>
 member function which returns an object (usually itself) which has a
 <code>next()</code> member function.   The <code>next()</code> method is
 then invoked till it raises a <code>StopIteration</code> exception.
@@ -206,6 +206,8 @@

 </pre>

+<h3>Wrapping Iterator</h3>
+
 <p>The problem with this approach, is that a producer could potentially
 block, and if it did, the entire process could potentially stop servicing
 other requests.   Thus, some mechanism for pausing the flow and rebuilding
@@ -259,11 +261,13 @@

 </pre>

-<p>This seems like quite the effort, wrapping each iterator and
+<h3>Cooperate</h3>
+
+<p>This seems like quite the effort, wrapping each iterator and
 then having to alter the calling sequence.  Why?  The answer is
 that it allows for a <code>flow.Cooperate</code> object to be
 returned.   When this happens, the entire call chain can be
-moved off the stack so that other flows can proceed.   For
+moved off the stack so that other flows can proceed.   For
 flow.Iterator (which blocks), the implementation of Cooperate
 simply puts the call chain to sleep</p>

@@ -281,10 +285,13 @@
 #   3
 </pre>

+
+<h3>Merging Iterators in one flow</h3>
+
 <p>Perhaps a more clear example can be found when using the
 Merge function in Flow.  This simply zips two or more wrapped
 iterators together, without blocking one or the other.  In the
-example below, the <code>States</code> iterator isn't blocked
+example below, the <code>States</code> iterator isn't blocked
 by the <code>Counter</code> iterator.
 </p>

@@ -324,7 +331,7 @@
 <code>reactor.callLater</code> and <code>internet.defer.Deferred</code>
 mechanism, things are very nice.  In the example below, the first two
 items in the list are produced (although they arn't delivered yet),
-other events in the reactor are allowed to proceed, and then the
+other events in the reactor are allowed to proceed, and then the
 last item in the list is produced.</p>


@@ -333,7 +340,7 @@
 from twisted.internet import reactor
 import flow

-def prn(x):
+def prn(x):
     print x
 d = flow.Deferred([1,2,flow.Cooperate(1),3])
 d.addCallback(prn)
@@ -344,6 +351,48 @@
 #   [1,2,3]
 </pre>

+<h3>Using database connexion to prodive data</h3>
+
+<p>
+Of course, Flow allows to use ConnectionPools to provide data.<br />
+QueryIterator is a simple Iterator which provides data from a database. Is
uses <code>twisted.enterprise.adbapi</code>
+connectionPools, and a query to fetch data. QueryIterator must be used with
a deferred execution.
+</p>
+
+<pre class="python">
+from __future__         import generators
+from twisted.enterprise import adbapi
+from sandbox.flow       import QueryIterator
+from twisted.internet   import reactor
+
+import flow
+
+dbpool = adbapi.ConnectionPool("MySQLdb",host='localhost',
db='ADB',user='AUSER',passwd='APASSWD')
+sql = """SELECT name FROM mytable LIMIT 0,3"""
+
+def consumer():
+    query = flow.Wrap(QueryIterator(dbpool, sql))
+    while 1:
+        try:
+            yield query
+            if query.stop: break
+            print "Processed result : ",query.result
+        except flow.StopIteration: print "-- Finished --"
+
+from twisted.internet import reactor
+def finish(result): print "Final result : ", result
+f = flow.Deferred(consumer())
+f.addBoth(finish)
+reactor.callLater(1,reactor.stop)
+reactor.run()
+
+# prints
+# Processed result :  ('SMITH',)
+# Processed result :  ('JOHN',)
+# Processed result :  ('BOB',)
+# Final result :  []
+
+</pre>

 </body>
 </html>