[Twisted-Python] ADBAPI and individual rows

[Dave Peticolas]

On Fri, 2003-08-08 at 16:12, Edmund Dengler wrote:
> Greetings!
> 
> Digging a bit more into the DB side, and I wonder a bit about the API for
> adbapi. From what I can tell, the runQuery() will return a Deferred for
> the whole result set (which implies the result set is in memory). Is there
> a method or style for running a query in which each row will be passed
> back through a Deferred one at a time? (Basically, each cycle through
> reactor.run() would be returning a single row).
> 
> What I am looking for is how to deal with a large result set without
> requiring the whole set be in memory. Should I be looking elsewhere within
> Twisted to do this?

You should look at runInteraction. That calls a user-specified
function with what is essentially a DB-API cursor in a separate
thread. You can do what you like with the cursor, including
fetching results one row at a time.

You might also look at the Flow module. I've never used it,
but I think it was designed to solve such problems.

dave

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: </pipermail/twisted-python/attachments/20030809/c72b29d4/attachment.sig>