Thanks everyone for your comments, you were all a lot of help.<br><br>Andrew, I was aware that the deferred didn't go anywhere, I oversimplified things in the example, in the original design in my head, the code that actually does the db calls was waiting on the deferred. Actually, the way it's written, it never even runs, I left out the first call to processChunk() necessary to start the iteration. But your comments helped a lot.
<br><br>My concern wasn't ElementTrees built in stuff, but 300+ line spaghetti procedure that massages the element tree data into a dictionary. That needs to be rewritten anyway.<br><br>I messed around this morning, and here's what I have done. I compiled cElementTree onto the test box. The LineReceiver uses
cElementTree.XMLTreeBuilder.feed() and builds the tree on the fly.<br><br>At connectionClose it calls a method that runs the business logic routine in a thread, using <span class="py-src-variable">threads</span>.<span class="py-src-variable">
deferToThread, and adding a callback; said callback executes the db code. That still blocks for now.<br><br>So, my current plan is to refactor the tree-->dict code, but call that in a separate thread. Rewite the dict --> db layer. The only question I have, is since it's a long series of insert/update calls in which I don't need results other than, 'it worked' , is, after reading the comments, it seems like I might just best be served by writing a series of straight dbi calls; and wrapping that all in one call to
thread.callInThread. <br><br>The db is MySQL, and feeds a PHP-based reporting app, so I can't use pgasync. <br><br>Thanks again all.<br></span>