[Twisted-Python] application design / threads

[Henning.Ramm at mediapro-gmbh.de]

Hi again!

Being a bloody newbie I'm quite unsure how to construct my application; twisted has so much possibilities, perhaps you can provide some hints.

The whole thing becomes a workflow manager that collects information about processed newspaper pages. I'm planning to connect several modules (running on different machines) with PB. The server will handle a database connection and perhaps cache some data that the clients might request. Later the server will provide some status information via http (praise twisted).

One part is an Acrobat Distiller wrapper (via win32com.client):

I created a library with makepy and implemented an event handler class that catches Distiller's events, e.g. "OnJobStart".
That handler class is inherited from a base class by makepy. Each event calls one predefined method.
The handler class is connected with the distiller class like
dist = acrobatdistiller.PdfDistiller()
win32com.client.WithEvents(dist, DistEventHandler)
There can be only one Distiller instance. I guess I should keep that as long as possible, for several files to distill.
-> How should I handle these events with twisted? Do I need multiple inheritance to inherit my handler also from some twisted class?

Before a PostScript file gets distilled [dist.FileToPDF(source, target, joboptions)], I must patch it (write some pdfmarks into it). I guess I should use a thread for those blocking operations (distilling can take several minutes).
-> One thread for both, just work blocking?
-> Do I need a thread for every copy/move operation (large files via network)?

Another part reads some XML files (using minidom) and writes the mangled content into a database.
I guess I can't split XML parsing into smaller non-blocking portions -> one thread for every file?
For the database stuff I'm using runInteraction, seems to be the best solution.

A general issue is logging. I'm used to log4perl/log4net (same as log4j from apache), but dropped log4py in favour of the similar Python 2.3 standard logging. I already asked in my last mail to this list how I can integrate that with twisted. 
-> Is it possible to get correct linenumbers if using deferreds? Or can I trust logging to be non-blocking?


Best regards,
Henning Hraban Ramm
Südkurier Medienhaus / MediaPro
Support/Admin/Development Dept.