tuning riak for 100s of concurrent connections

Wilson MacGyver wmacgyver at gmail.com
Fri Feb 25 13:06:26 EST 2011


any advise to solve this?

On Fri, Feb 25, 2011 at 1:05 PM, Sean Cribbs <sean at basho.com> wrote:
> Yes, the majority of the cost is probably in TCP setup. That pain is happening at the TCP stack level, not in Erlang.  It's actually really cheap and easy to spawn a new process in Erlang, so there's no reason to have "standby workers", especially when they don't need to maintain state between requests.
>
> Sean Cribbs <sean at basho.com>
> Developer Advocate
> Basho Technologies, Inc.
> http://basho.com/
>
> On Feb 25, 2011, at 12:45 PM, Wilson MacGyver wrote:
>
>> I've tried it, it didn't have much impact. A bit more info on how I'm
>> doing the test.
>>
>> I'm using apachebench. I purposely have it grab the same key/value over and over
>> again.
>>
>> if I use concurrent connection of 10, 1000 requests each. 50% of the
>> request complets
>> within 7ms, longest request is 38ms. this is quite good.
>>
>> now, if I increase the concurrent connection to 100, 50% of the
>> request complets at 77ms.
>>
>> it "feels" like a paying for startup cost of connection problem to me.
>> Is there a way to purposely
>> startup riak with a bunch of standby workers, or something to that effect?
>>
>> On Fri, Feb 25, 2011 at 11:37 AM, Sean Cribbs <sean at basho.com> wrote:
>>> You can disable Nagle on the riak side (at least on 0.14 and later). Put this in the riak_core section of app.config:
>>>
>>> {disable_http_nagle, true}
>>>
>>> Sean Cribbs <sean at basho.com>
>>> Developer Advocate
>>> Basho Technologies, Inc.
>>> http://basho.com/
>>>
>>> On Feb 25, 2011, at 9:33 AM, Wilson MacGyver wrote:
>>>
>>>> TCP_NODELAY is something you'd set when you use the socket API call,
>>>> not a global tuning setting on the OS as I recall.
>>>>
>>>> On Fri, Feb 25, 2011 at 9:28 AM, Nico Meyer <nico.meyer at adition.com> wrote:
>>>>> Whenever I see latencies which are roughly multiples of 40ms it screams
>>>>> to me 'nagle algorithm'. I have seen this so often now, that the first
>>>>> thing I check is, if the TCP_NODELAY option is set on the TCP socket on
>>>>> both ends.
>>>>
>>
>> --
>> Omnem crede diem tibi diluxisse supremum.
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>



-- 
Omnem crede diem tibi diluxisse supremum.




More information about the riak-users mailing list