tuning riak for 100s of concurrent connections

Nico Meyer nico.meyer at adition.com
Fri Feb 25 14:43:11 EST 2011


Just out of curiosity I did some tests myself on one of our production
machines. We normally only use the ProtocolBuffers interface and a
thrift interface that we wrote ourselves.

If I fetch a small key (~250 bytes), the riak server becomes CPU bound
with about 20 concurrent requests, at which point the latency naturally
becomes larger. At this point one riak server is handling over 6000
requests/s. This is on an 8 core system (Dual Intel Xeon E5506
@2.13GHz), and all cores are at nearly 100%. No tuning was done on
either riak or the OS.

Also 99% of the requests still took less then 20ms, while with 10
concurrent requests its more like 2ms.


Am Freitag, den 25.02.2011, 13:54 -0500 schrieb Wilson MacGyver:
> SO_REUSEADDR is also something you set at the socket API as I recall.
> So I don't think it's something you can just set on the TCP/IP itself
> as a global setting.
> 
> On Fri, Feb 25, 2011 at 1:41 PM, Les Mikesell <lesmikesell at gmail.com> wrote:
> > Those settings shouldn't make a big difference in how the number of
> > connections scale up, though.  There is a theoretical maximum rate limit for
> > creating new connections as each socket is supposed to sit in TIME_WAIT for
> > a packet round-trip time to ensure that nothing outstanding will collide
> > with that socket number when it is reused for the same IP address.  Maybe
> > your test is hitting that limit.  Can you set SO_REUSEADDR?
> 
> 






More information about the riak-users mailing list