[Twisted-web] Some help with Nevow and databse I/O

[Eric Gaumer]

Hello everyone,

Twisted is relatively new to me but I've become fascinated by the idea of 
asynchronous network programming and I generally enjoy the feel of the 
twisted framework.

I'm developing a web application with a limited set of pages that will 
eventually serve as a bandwidth statistics application (uses SNMP to poll 
switch counters).

I'm using newvow to render some simple XML pages which basically take the form 
of:

    def data_listCusotmers(self, context, data):
        args = inevow.IRequest(context).args
        db = shelve.open('authdb')
        return db

    docFactory = loaders.xmlfile('templates/ListCustomers.html')

I have two questions that are somewhat interrelated. In my test cases I have 
authentication (using nevow session guard) setup using a SHA stored via the 
shelve module. 

The true customer info is stored in a MySQL database. I can pull that data 
doing the following:

    def data_query(self, context, data):
        return self.dbpool.runQuery('SELECT name, email, id, TransferType, 			
		basic_charge, over_charge, permitted_transfer FROM Customers')

I've read Abe's book on twisted and on page 54 he states "Nevow is designed 
from the ground up for Twisted, which means you can use Deferreds everywhere"

My question is do I have to use the adbapi module or can I use deferreds to 
handle database queries?

I've read the docs on deferreds but I'd be lying if I said I can fully wrap my 
brain around all aspects of the concept or how it's implemented.

I don't understand why I would need separate threads in the first place. I 
know that the query would block but isn't that transaction being done via 
sockets anyway? Why can't I use the same mechanism (deferred) I would use, to 
say, defer work waiting on a response from a mail server? Can't I submit a 
query, have the reactor go do something else, and have a callback run when 
the data becomes available? Why do I need to spawn threads to avoid blocking?

It should be pretty obvious that I don't fully grasp the concept of 
asynchronous socket programming. Can somebody provide insight that may help 
me to better understand things.

I can live with using the adbapi but wonder if it's possible to simply defer 
that work and use a callback mechanism (I'm using the original twisted.web 
not twisted.web2).

I've also read Glyf's post where he states "The documentation is really poor, 
and never says this, but using Twisted, or rather ADBAPI, to convert every 
single SQL statement into a separate transaction and handle its results 
separately, has a whole slew of problems".

http://glyf.livejournal.com/2005/09/24/

I was pretty much heading down this road but I don't really understand how 
else to do it. I've browsed many examples and have never really come across 
one that involves session.guard, newvow, and db access all in the same 
application. Anyone have any experience they could share?

Thanks, 

-- 
Eric Gaumer
Debian GNU/Linux PPC
egaumer at pagecache.org
http://egaumer.pagecache.org
PGP/GPG Key 0xF15D41E9
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://twistedmatrix.com/pipermail/twisted-web/attachments/20060224/273e48e7/attachment.pgp