[Twisted-Python] Woven example for article, Part II

[Dr. David Mertz]

Hi again,

I wrote a note a bit ago about my problems running the 
PicturePile example.  But that's not really what I would most 
want to present in my article anyway.

In a tip I wrote for the IBM developerWorks XML Zone, I developed 
a slightly-better-than-trivial example of a remote monitor of 
hits to a website.  The initial custom client/server used XML as 
a line format, but that's not important here.  I often like to 
continue an example over various iterations.

In particular, you can see that I reimplemented this same 
client/server (with slight variations in operation) in Twisted 
for my first article:

  http://gnosis.cx/secret/twisted1.txt

I'd like to continue this idea with more sophisticated--and web 
based--implementations.  One really simple approach is to simply 
create a static page that displays hits, but use the <meta> 
refresh tag to make it load repeatedly.  Meanwhile, a 
long-running script can update this page in the background.  In 
Twisted, it could be:

----------------------------------------------------------------------------
[gnosis at boa twisted]$ cat tlogmaker.py
#!/usr/bin/env python2.2
from twisted.internet import reactor
from webloglib import log_fields
from webloglib import ip, timestamp, request, status, bytes, referrer, agent

LOG = open('access-log')
RECS = []
PAGE = 'www/refresher.html'
TOP, ROW, END, COLOR = ...

def update():
    global RECS
    page = open(PAGE,'w')
    RECS.extend(LOG.readlines())
    RECS = RECS[-35:]
    print >> page, TOP
    odd = 0
    for rec in RECS:
        hit = [field.strip('"') for field in log_fields(rec)]
        if hit[status]=='200' and hit[referrer]!='-':
            resource = hit[request].split()[1]
            print >> page, ROW % (COLOR[odd], hit[referrer], resource)
            odd = not odd
    print >> page, END
    page.close()
    reactor.callLater(5, update)

update()    
reactor.run()
----------------------------------------------------------------------------

Of course, this is completely trivial as a use of Twisted--using 
time.sleep() in the loop would be exactly as good (and I made a 
version that does that for comparison).  The produced page, btw, 
is at:

  http://gnosis.python-hosting.com/refresher.html

But the update script may or may not be running while you check 
the page (which doesn't affect the viewability of the page, just 
its up-to-date-ness.

The next step is probably to show a 'refresher.rpy' variant on 
the same basic thing.  That is straigtforward, and I understand 
how to do that.

But past that, I'd like to create a version that has more 
meaningful interaction with the web browser.  One obvious 
interaction is to let the client select which log fields to 
display--probably using a bunch of checkboxes for those fields 
listed in the second 'from weblog import ...' line of the above 
script.  Maybe this user preference could be stored in a cookie 
(maybe number of rows to display as another option).

I don't really understand Twisted well enough to know what the
most logical approach to this next variant would be.  It seems
like woven is the main web technology in Twisted... but given my
problems running even the bundled example, I'm not sure how to
proceed.

If anyone here would like to help me create an example, that 
would make my next article better--and I'll give you credit for 
the assist.  I like Twisted, but with all the pieces, its hard to 
get started nicely... maybe a good few articles (by me, or 
otherwise) can help bridge the gap for new users (like me :-)).

Yours, David...