[Twisted-Python] Twisted webserver performance issues

[exarkun at twistedmatrix.com]

On 02:55 am, leyssw at ihug.co.nz wrote:
>Thanks for the link info.
>
>I literally just set up Varnish for the first time on my web host, so I 
>am no pro. However, varnish uses a c like scripting language that can 
>run regexps on the incoming urls and apply rules to them.
>
>What this means is that we can (as a starting point) choose caching 
>times for each of those high traffic urls in the varnish script without 
>touching trac. In the script, we match the front page (with a regexp) 
>and add a max- cache time to 86400, cache it, etc.
>
>See http://www.catalystframework.org/calendar/2008/14
>for a great varnish tutorial
>
>By doing that for each of the urls you listed (bar the timeline, which 
>should be cached for 1 hour or perhaps 30 minutes instead) we will 
>seriously lighten the load on the database and the webserver, even if 
>varnish runs on the same machine (as a start).
>
>Every other url (which doesn't have caching set) we could just cache 
>for 30 minutes (perhaps even a day), but only for anon users (no 
>cookies set in browser)

Where this gets hard is that the front page looks different depending on 
whether the person requesting it is anonymous or authenticated.  If 
they're anonymous, various links are not presented.  If they're 
authenticated, they get the extra links and their username is part of 
the page.

Jean-Paul