[Twisted-Python] Cause epollreactor to busyspin
Tim Hughes
thughes at thegoldfish.org
Thu Apr 21 11:26:48 MDT 2016
On 2016-04-21 11:30, Tim Hughes wrote:
> On 2016-04-21 03:57, Glyph wrote:
>>> On Apr 20, 2016, at 10:37, Tim Hughes <thughes at thegoldfish.org>
>>> wrote:
>>>
>>> Hi All,
>>>
>>> I have a latency sensitive application that gets affected when it is
>>> moved between cores or the system has to wake from idling.
>>
>> Can you explain "gets affected" in more detail? If moving between
>> cores is the issue, could you set its CPU affinity?
>
> That is exactly what I would do with a physical machine but
> unfortunately where I need to use it is in china and I only have xen
> virtual machines available. I know this isn't the best situation but
> unfortunately that is all I can get in these locations.
>>
>>> I would like to run the thread in as tight a loop as possible and to
>>> do this i need to cause epollreactor to busyspin on receiving data.
>>
>> I am very curious about your application now :)
>
> We are using it to test the performance of streaming financial market
> data over different CDN providers to different cities around the
> world. Currently trying to jump through the hoops to be able to
> opensource it and stick it on github. It isn't that exciting, it is
> basically a modified LineReceiver with the delimiter set to \x01 and
> the messages are key=value pairs at a peak rate of approx 130000 pairs
> per second
> https://en.wikipedia.org/wiki/Financial_Information_eXchange
>
Hoops have been jumped and it is on github at
https://github.com/LMAX-Exchange/txfixclient
>>
>>> I have worked out how to do it by editing the twisted code directly
>>> by hard coding the value of `timeout` to be 0 at this location in the
>>> code
>>> https://github.com/twisted/twisted/blob/f074ba3d5083aa1503abcf194aece327e7f84805/twisted/internet/epollreactor.py#L370
>>
>>> Is it possible to actually set this on the reactor in a more sensible
>>> way so I don't need to patch the twisted codebase. ? Below is
>>> basically what I am doing.
>>
>> You can achieve this with a tiny bit of extra overhead by doing
>> something like this:
>>
>> def cant_sleep_clown_will_eat_me():
>> reactor.callLater(0, cant_sleep_clown_will_eat_me)
>> cant_sleep_clown_will_eat_me()
>
> Will give this a go
>
>>
>> Does this actually improve your latency?
>>
>> -glyph
>> _______________________________________________
>> Twisted-Python mailing list
>> Twisted-Python at twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
More information about the Twisted-Python
mailing list