[Twisted-Python] Twisted webserver performance issues

[James Y Knight]

On Nov 8, 2009, at 9:02 AM, exarkun at twistedmatrix.com wrote:

> On 04:26 am, foom at fuhm.net wrote:
>> When I last looked into the performance issues, I found that  
>> sometimes
>> trac appears to block for long periods of time without releasing the
>> GIL. That seems to be the core of the performance issues, currently.
>> When it's responding normally, it's perfectly snappy. But, sometimes,
>> it blocks for 10sec at a time.
>
> How were you detecting these times?

I first saw it while reloading the website and watching the resource  
loading timeline in Safari. And then after I pulled out the static  
resources from Trac's threads (which seems to have helped  
substantially for the /usual/ case), sometimes even those would hang  
for 10 seconds unresponsive. So then I had a loop running curl; sleep  
2 on Cube, and noticed it happening periodically.

I don't know for sure it's a GIL-releasing issue, it might be that  
something actually hung up the twisted mainloop thread, but I'd bet 5  
cookies it was one of the trac threads blocking doing something with  
the GIL held.

I didn't get farther in the investigation than this.

> Can we attach gdb to the process and trigger an all-threads stack  
> dump when it happens?

That's a good idea. Unfortunately, currently it seems to be working  
quite smoothly. :)

James