[Twisted-Python] suggestion for tracking down cause of 2 bugs sought: possibly threads / defer.DeferredSemaphore ?

[Jonathan Vanasco]

On Mar 21, 2007, at 11:32 AM, Jonathan Vanasco wrote:


> 	2)
> 		I'm experiencing an obscene amount of process growth.  I'm fairly  
> certain that its from constant errors in urllib2 , which has been  
> nothing of a nightmare with countless known bugs and patches  
> ( downloading takes place via Mechanize , which is wrapped via  
> urllib2 ), and makes me consider just wrapping wget or curl.   but  
> this could very well be threading related.
> 		
> 		Question:
> 			Is there any way to profile memory usage by twisted in a  
> granular fashion ?  I'd like to figure out where this memory usage  
> is ( it grows steadily to ~1.7gb after 8 hours ).
>

I just came across this:
	From: 	  mwl at point45.com
	Subject: 	Re: [Twisted-Python] questions about large db queries and  
memory leak
	Date: 	February 8, 2007 8:28:59 AM EST
	...
	If you're not running Python >= 2.5.0 this might be relevant:
	http://evanjones.ca/python-memory.html


i was on 2.4, upping to 2.5 solved that issue.  incremental growth  
of .2mb / thread instance ( which ate 1.7gb of ram in an hour or  
so ) , has been replaced by a steady 18-25mb daemon

my machine is responsive again.

// Jonathan Vanasco

| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -  
- - - - - - - - - - - - - - - - - - -
| SyndiClick.com
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -  
- - - - - - - - - - - - - - - - - - -
|      FindMeOn.com - The cure for Multiple Web Personality Disorder
|      Web Identity Management and 3D Social Networking
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -  
- - - - - - - - - - - - - - - - - - -
|      RoadSound.com - Tools For Bands, Stuff For Fans
|      Collaborative Online Management And Syndication Tools
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -  
- - - - - - - - - - - - - - - - - - -