[Twisted-Python] Question regarding the working of twisted python with celery
exarkun at twistedmatrix.com
exarkun at twistedmatrix.com
Wed Mar 2 06:34:35 MST 2011
On 09:42 am, dineshkapoor27 at gmail.com wrote:
>Hi,
>
> I have run into a weird performance issue while working with twisted
>and
>using celery to schedule tasks on it. Here is my setup:
>
>1. Celery schedules a task which makes a TCP connection to a server
>running
>FreeSwitch
>2. Celery is running with concurrency = 2.
>3. I have changed my code so that I call reactor.callfromthread for all
>reactor based work.
>4. A lot of times once I schedule the celery task, I get delays ranging
>from
>3 - 30 seconds when I am running with Celery concurrency =2.
>5. If I reduce the celery concurrency to 1, then everything works
>great!
>6. Upon debugging it a little bit more, it seems that the
>selectReactor's
>doSelect is where the code is getting stuck on the select().
>
>I dont know how to get around the delay, and I dont know who is causing
>that
>delay exactly. If there is more info that is needed then please let me
>know.
>Everything works fine when the Celery worker process is 1, but delay
>starts
>happening invariably when celery's worker processes >=2.
I think Reza Lotun explained what's causing this problem.
If you haven't looked at it yet, you might want to see if Ampoule can
help you out.
Jean-Paul
More information about the Twisted-Python
mailing list