[Twisted-Python] Using Twisted for distributed computation / experiment running?

[robomancer]

> It seems you are wanting a hybrid between a batch queueing system and
> a data management system.
>
> Any reason not to use something like Torque (nee' OpenPBS) for the
> batch part?
>
> http://www.clusterresources.com/pages/products/torque-resource-manager.php

Thanks for the references.  Torque seems like a good possibility, but
a bit heavyweight for my tastes; I'd rather have something small and
flexible that I can easily edit to suit my needs.  That's why I'd
prefer a solution in Python (whether or not it involves Twisted).  I
really am looking only at small-scale stuff -- I have no need for
fault tolerance or scalability beyond maybe 5-10 nodes.

> More info on what your data is like is probably needed for ideas on
> the second part.

All I need is the ability to test several different algorithms on
several different input files. Each algorithm has a variety of
parameter settings, so for every experiment I need to record which
algorithm was used, the parameters, which input file was used, and the
quality of the result (from -1 to 1).  I don't think a data management
system is necessary here; I was basically planning on using .csv files
to store experiment settings and results.   Again, I'm aiming for
lightweight, not enterprise-grade :)