[Twisted-Python] Simultaneously client-server app

[Andrew Bennetts]

On Fri, Jul 11, 2003 at 04:55:23PM +1000, Minh Luu wrote:
>    Twisted Masters,
>     
>    Has anyone any suggestion on how I might start on the following task using
>    Twisted?
>     
>    What I would like to do is a type of server that will serve a number of
>    clients and itself is a client of other servers. It requires a spool of
>    worker threads to perform time-consuming tasks and database related
>    tasks.  All the network I/O will be handled by a single async thread using
>    select. A simple diagram below demo the model I would like to implement.
[snip pretty diagram]

I doubt threads and select are actually requirements you have, rather, it's
just how you think it should work.  But if you know how it should work, why
ask us?  ;)

There are lots of blanks in your description, so it's hard to guess what you
need to know.  You effectively have asked a couple of very general
questions, so I'll give some general answers, and hope that's enough.  If it
isn't, try being more specific about what you're doing, and I'll try to give
more specific answers.


* Question: *
"What I would like to do is a type of server that will serve a number of
clients and itself is a client of other servers."

* Answer: *
Network clients and servers in Twisted are pretty easy, and its very good
at them -- have you seen these HOWTOs?
    http://twistedmatrix.com/documents/howto/servers
    http://twistedmatrix.com/documents/howto/clients

Being a server and a client at once is as easy as:
    reactor.listenTCP(1234, MyServerFactory())
    reactor.connectTCP('host', 4321, MyClientFactory())


* Question: *
"...a spool of worker threads to perform time-consuming tasks..."

* Answer: *
Have you seen the "Using Threads" HOWTO?
    http://twistedmatrix.com/documents/howto/threading

The key tool you probably want to use when working with threads is
twisted.internet.threads.deferToThread.

Also, be aware that threads aren't necessarily the only solution to
"time-consuming tasks", although it would help to know what these tasks
actually are :)


* Question: *
"...and database related tasks"

* Answer: *
The twisted.enterprise package has support for using standard Python DB-API
2.0 compliant modules.  You probably just want to use
twisted.enterprise.adbapi, which is described here:
    http://twistedmatrix.com/documents/howto/enterprise

That document is a little out of date, unfortunately -- it doesn't mention
that you can avoid adbapi.Augmentation if you don't like that approach, and
just use adbapi.ConnectionPool directly, although the its API is fairly ugly
and not very Twisted-like.  That module could probably do with an overhaul,
but that's not your problem ;)


I hope I've helped at least a little.

Regards,

-Andrew.