tuning riak for 100s of concurrent connections

Sean Cribbs sean at basho.com
Fri Feb 25 13:25:56 EST 2011


Now you're talking about tweaking kernel-level settings -- something well outside my expertise (although TCP slow-start comes to mind).  That said, it's kind of hard not to use TCP if you want a reliable connection, warts and all.

Sean Cribbs <sean at basho.com>
Developer Advocate
Basho Technologies, Inc.
http://basho.com/

On Feb 25, 2011, at 1:06 PM, Wilson MacGyver wrote:

> any advise to solve this?
> 
> On Fri, Feb 25, 2011 at 1:05 PM, Sean Cribbs <sean at basho.com> wrote:
>> Yes, the majority of the cost is probably in TCP setup. That pain is happening at the TCP stack level, not in Erlang.  It's actually really cheap and easy to spawn a new process in Erlang, so there's no reason to have "standby workers", especially when they don't need to maintain state between requests.
>> 
>> Sean Cribbs <sean at basho.com>
>> Developer Advocate
>> Basho Technologies, Inc.
>> http://basho.com/
>> 
>> On Feb 25, 2011, at 12:45 PM, Wilson MacGyver wrote:
>> 
>>> I've tried it, it didn't have much impact. A bit more info on how I'm
>>> doing the test.
>>> 
>>> I'm using apachebench. I purposely have it grab the same key/value over and over
>>> again.
>>> 
>>> if I use concurrent connection of 10, 1000 requests each. 50% of the
>>> request complets
>>> within 7ms, longest request is 38ms. this is quite good.
>>> 
>>> now, if I increase the concurrent connection to 100, 50% of the
>>> request complets at 77ms.
>>> 
>>> it "feels" like a paying for startup cost of connection problem to me.
>>> Is there a way to purposely
>>> startup riak with a bunch of standby workers, or something to that effect?
>>> 
>>> On Fri, Feb 25, 2011 at 11:37 AM, Sean Cribbs <sean at basho.com> wrote:
>>>> You can disable Nagle on the riak side (at least on 0.14 and later). Put this in the riak_core section of app.config:
>>>> 
>>>> {disable_http_nagle, true}
>>>> 
>>>> Sean Cribbs <sean at basho.com>
>>>> Developer Advocate
>>>> Basho Technologies, Inc.
>>>> http://basho.com/
>>>> 
>>>> On Feb 25, 2011, at 9:33 AM, Wilson MacGyver wrote:
>>>> 
>>>>> TCP_NODELAY is something you'd set when you use the socket API call,
>>>>> not a global tuning setting on the OS as I recall.
>>>>> 
>>>>> On Fri, Feb 25, 2011 at 9:28 AM, Nico Meyer <nico.meyer at adition.com> wrote:
>>>>>> Whenever I see latencies which are roughly multiples of 40ms it screams
>>>>>> to me 'nagle algorithm'. I have seen this so often now, that the first
>>>>>> thing I check is, if the TCP_NODELAY option is set on the TCP socket on
>>>>>> both ends.
>>>>> 
>>> 
>>> --
>>> Omnem crede diem tibi diluxisse supremum.
>>> 
>>> _______________________________________________
>>> riak-users mailing list
>>> riak-users at lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> 
>> 
> 
> 
> 
> -- 
> Omnem crede diem tibi diluxisse supremum.
> 
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com





More information about the riak-users mailing list