[Twisted-Python] twisted.web and MySQLdb

[Moshe Zadka]

I won't reply to your question here, I'm sure someone more competent
than me can do so.
I will take a minute to note that for some reason, the instinct of
web developers has been to immediately write applications in the
LAMP model. While twisted.web is a perfectly good substitute for
A, and can indeed be used this way, it is much more capable.

LAMP -- Linux Apache MySQL PHP [each of those has variants]

Linux (and most other things in the "L" place, such as FreeBSD) has
an incredibly fast fork() operation. This leads people to write their
web code with the process-per-page model. Apache optimizes this by
preforking, making such code actually scalable enough to deliver good
response times. However, suddenly there is a problem: since different
processes are likely to deal with the same user, memory becomes fragile.
It is unsafe to put things in memory: processes switch users, get born
and die too often. In Linux, and most other Unices, file locking has
been a thorny issue. So instead of putting data in files, where the
subtle semantics of locking and concurrency should be dealt with,
data tends to end up in the database. That means *all* data. Sessions.
Temporary "show this message when the user finished his redirect loop".
Etc. etc. Once all the data is in MySQL, which is the easiest database to
set up, it becomes natural for the average page to be a "glorified select".
PHP was designed for this exact scenario: take a select table and spruce
it up with HTML.

Unfortunately, what LAMP deals to is exactly to this: inevitably, all
pages are just spruced up select-tables. This makes programming somewhat
unnatural unless you're programming a database viewer.

With twisted.web, you don't have these problems. It is easy to keep data
in files, because locking is not an issue. It is easy to cache data in memory
because everything is served from a single process. If you need to attach
state to a user's session, you can just keep an object in the session.
This means programming twisted.web should be a lot more like writing
a GUI application, and a lot less like writing select-with-HTML.
You should probably reconsider whether you really want MySQL. It adds
complexity to your application, and the gain is usually small. Putting
persistent data in files, and using liberal caching schemes, also plays
on the core advantage of Linux (and similar) -- it uses the incredibly
optimized caching algorithms.