[Twisted-Python] twisted.web and MySQLdb

[phil at bubblehouse.org]

well, my only use of the db object directly is for actually updating the
table. my particular implementation would have far more views than
updates, so i was going to cache the common data, or possibly pre-render
html...

wouldn't you say that these performance issues are as much of a
consequence of poor design as much as the choice to use SQL? obviously db
calls are expensive...but some of the benefits of a portable datasource
are worthwhile. as nice as it would be to live in a world where pickle was
a common data format, we're not there yet...


> Date: 29 Oct 2003 06:28:17 -0000
> From: Moshe Zadka <m at zadka.site.co.il>
> Subject: Re: [Twisted-Python] twisted.web and MySQLdb
> To: twisted-python at twistedmatrix.com
> Reply-To: twisted-python at twistedmatrix.com
>
> I won't reply to your question here, I'm sure someone more competent
> than me can do so.
> I will take a minute to note that for some reason, the instinct of
> web developers has been to immediately write applications in the
> LAMP model. While twisted.web is a perfectly good substitute for
> A, and can indeed be used this way, it is much more capable.
>
> LAMP -- Linux Apache MySQL PHP [each of those has variants]
>
> Linux (and most other things in the "L" place, such as FreeBSD) has
> an incredibly fast fork() operation. This leads people to write their
> web code with the process-per-page model. Apache optimizes this by
> preforking, making such code actually scalable enough to deliver good
> response times. However, suddenly there is a problem: since different
> processes are likely to deal with the same user, memory becomes fragile.
> It is unsafe to put things in memory: processes switch users, get born
> and die too often. In Linux, and most other Unices, file locking has
> been a thorny issue. So instead of putting data in files, where the
> subtle semantics of locking and concurrency should be dealt with,
> data tends to end up in the database. That means *all* data. Sessions.
> Temporary "show this message when the user finished his redirect loop".
> Etc. etc. Once all the data is in MySQL, which is the easiest database to
> set up, it becomes natural for the average page to be a "glorified
> select".
> PHP was designed for this exact scenario: take a select table and spruce
> it up with HTML.
>
> Unfortunately, what LAMP deals to is exactly to this: inevitably, all
> pages are just spruced up select-tables. This makes programming somewhat
> unnatural unless you're programming a database viewer.
>
> With twisted.web, you don't have these problems. It is easy to keep data
> in files, because locking is not an issue. It is easy to cache data in
> memory
> because everything is served from a single process. If you need to attach
> state to a user's session, you can just keep an object in the session.
> This means programming twisted.web should be a lot more like writing
> a GUI application, and a lot less like writing select-with-HTML.
> You should probably reconsider whether you really want MySQL. It adds
> complexity to your application, and the gain is usually small. Putting
> persistent data in files, and using liberal caching schemes, also plays
> on the core advantage of Linux (and similar) -- it uses the incredibly
> optimized caching algorithms.