[Twisted-Python] Performance issue in reactor.callLater
Stefan Behnel
behnel_ml at gkec.informatik.tu-darmstadt.de
Mon Sep 6 08:29:55 MDT 2004
Itamar Shtull-Trauring schrieb:
> I would like to see some benchmarks comparing
> speed of adding and cancelling for cases with many scheduled calls, but
> also for cases with few calls, so we can see if your patches slow down
> other cases.
For the first part, I did a few profiler runs that showed results similar to the following. Please note that although I removed a few lines that only belong to my personal code, the top three lines actually /were/ the top three. The "AckManager" lines are a simplified form of my own implementation, see it as a second level callLater. The __run_callbacks method is so slow because it calls "reactor.callLater" to be run repeatedly.
---------------------------------------
9577183 function calls (9577108 primitive calls) in 117.349 CPU seconds
Ordered by: internal time, call count
ncalls tottime percall cumtime percall filename:lineno(function)
506487 22.837 0.000 30.519 0.000 bisect.py:3(insort_right)
50830 12.002 0.000 103.742 0.002 base.py:400(runUntilCurrent)
506487 8.974 0.000 47.078 0.000 base.py:362(callLater)
498759 8.865 0.000 66.300 0.000 AckManager.py:81(__run_callbacks)
50830 8.700 0.000 8.911 0.000 default.py:472(doSelect)
3922829 7.682 0.000 7.682 0.000 base.py:132(__lt__)
506487 7.585 0.000 7.585 0.000 base.py:54(__init__)
500901 5.998 0.000 5.998 0.000 AckManager.py:59(__remove_inactive_entries_from_queue)
500181 5.953 0.000 51.499 0.000 AckManager.py:68(__schedule_run)
1 2.463 2.463 117.348 117.348 default.py:127(mainLoop)
50830 2.232 0.000 2.232 0.000 base.py:391(timeout)
---------------------------------------
I think it is actually easy to see here that bisect becomes a serious bottleneck when the list gets longer. I didn't include any results from cancel() as I immediately eliminated all calls to the method after my first profiler run. Believe me, it was absolutely worth it.
I'll just continue reimplementing the IReactorTime stuff and then re-run the profiler for you.
I'll also try to run a few test cases with cancel() - thanks to CVS, I should still have some old code around...
Stefan
More information about the Twisted-Python
mailing list