[Twisted-Python] Deffered question

[Matteo Giacomazzi]

Hi,

> I need to implement a command based protocol, that is a client sends
> a  command  to  the  server  and  waits  for the response. Since the
> response  computation  might  be  time  consuming  I  have to choose
> between 2 solutions:
> - use Twisted deffereds
> - or  write my own implementation, a TCP based threaded server. I'm
> not  worried about scaling since it will not be used by more than 10
> clients at a time.

I'm facing a problem like this.

I  used  to  have  a multithreaded server (fully written using Python)
that  accepted  commands  via  a  TCP  or  UNIX  socket, elaborate the
response  (tipically  querying  some remote servers via HTTP) and then
returned the response: one query, one thread.

This  server  runs  since  2001, I updated it, upgraded and refactored
some parts, but it's still kinda "heavy" to manage.
So, I realized that multithred is not _always_ the solution.

In  my case, the server was running (with a web server and a DB server
too) on a dual Xeon architecture. So, at most, two threads are running
on  the  same time but there are more chances that only one of them is
running while the other CPU is used by the web server or the DB.

That's the solution I'm building:
- there are different _processes_, one for every kind of remote server
I have to query;
- _processes_ may run on different (hardware) servers;
- processes  communicate via a  "message passing process" I wrote with
Twisted;

So:  one  process  accepts commands via TCP or UNIX socket, it quickly
parses  the  command and sends a message to the right ``server''; this
``server''  process  the command, does-whatever-it-needs and then send
the response to the first process that, in turn, quickly translates it
in a ``line oriented'' way and returns to the caller.

All  the servers are single-thread: that's because I need only to call
a  remote  host  and  to compute very very simple operations. Maybe in
your case threads would help - but how many threads?

I  would be interested in hearing opinion about this architecture from
Twisted experts.

Kindest regards,
 Matteo