Tuning a Riak cluster.

Richard Shaw richard at basho.com
Thu Feb 28 18:54:07 EST 2013


Not forgotten you Kevin, out of the country at the moment, will get back to
you soon as I can

Regards

Richard

On Tue, Feb 26, 2013 at 4:06 PM, Kevin Burton <rkevinburton at charter.net>wrote:

> Here is the config that I am using:****
>
> ** **
>
> {driver,****
>
>     basho_bench_driver_2i}.****
>
> ** **
>
> {operations, [****
>
>     {get_pb, 1},****
>
>     {{put_pb, 5}, 20},****
>
>     {{query_http, 10}, 1},****
>
>     {{query_mr,   10}, 1},****
>
>     {{query_pb,   10}, 1}****
>
> ]}.****
>
> ** **
>
> {measurement_driver,****
>
>     basho_bench_measurement_erlangvm}.****
>
> ** **
>
> {measurements, [****
>
>     {memory, 1000},****
>
>     {cpu, 1000},****
>
>     {processes, 1000},****
>
>     {filehandles, 1000}****
>
> ]}.****
>
> ** **
>
> %%% LOAD SETTINGS %%%****
>
> ** **
>
> {mode, max}.****
>
> {duration, 1}.****
>
> {concurrent, 3}.****
>
> ** **
>
> %%% DATA SHAPE %%%****
>
> ** **
>
> {key_generator, {uniform_int, 1000}}.****
>
> {value_generator, {fixed_bin, 1000}}.****
>
> ** **
>
> {pb_ips, ["172.16.33.107", "172.16.33.138", "172.16.33.221",
> "172.16.34.56"]}.****
>
> {pb_replies, 1}.****
>
> ** **
>
> {http_ips, ["172.16.33.107", "172.16.33.138", "172.16.33.221",
> "172.16.34.56"]}.****
>
> {http_port, 8098}.****
>
> ** **
>
> %%It can also be configured for devrel!****
>
> %%{pb_ips, [{"127.0.0.1", 8081},****
>
> %%          {"127.0.0.1", 8082},****
>
> %%          {"127.0.0.1", 8083},****
>
> %%          {"127.0.0.1", 8084}****
>
> %%]}.****
>
> %%{pb_replies, 1}.****
>
> %%****
>
> %%{http_ips, [{"127.0.0.1", 8091},****
>
> %%            {"127.0.0.1", 8092},****
>
> %%            {"127.0.0.1", 8093},****
>
> %%            {"127.0.0.1", 8094}****
>
> %%]}.****
>
> ** **
>
> {rng_seed, {1, 2, 3}}.****
>
> ** **
>
> %% enforce_keyrange is for use with sequential_int****
>
> %% Will error if keys in a range are missing.****
>
> %%{enforce_keyrange, 10000}.****
>
> ** **
>
> %%% MEASUREMENT SETTINGS %%%****
>
> ** **
>
> {nodes, ['riak at 172.16.33.107', ****
>
>          'riak at 172.16.33.138', ****
>
>          'riak at 172.16.33.221',****
>
>          'riak at 172.16.34.56']}.****
>
> {cookie, riak}.****
>
> ** **
>
> *From:* Richard Shaw [mailto:richard at basho.com]
> *Sent:* Tuesday, February 26, 2013 5:46 AM
>
> *To:* Kevin Burton
> *Cc:* riak-users
> *Subject:* Re: Tuning a Riak cluster.****
>
> ** **
>
> Hi Kevin,****
>
> ** **
>
> Yes I have, please can you let me know which basho bench config you're
> using and its contents****
>
> ** **
>
> Thanks****
>
> ** **
>
> Richard****
>
>
> ****
>
> ** **
>
> Richard Shaw****
>
> Client Services Engineer EMEA
> Basho Technologies
> +44 (0)7590 507683
> richard at basho.com****
>
> ** **
>
> On Mon, Feb 25, 2013 at 10:10 PM, Kevin Burton <rkevinburton at charter.net>
> wrote:****
>
> Did you have a chance to look at the basho_bench errors?****
>
>
> On Feb 23, 2013, at 2:35 AM, Richard Shaw <richard at basho.com> wrote:****
>
> ** **
>
> Hi Kevin,****
>
> ** **
>
> I sympathise with R, i'm going to write an installation guide to go on the
> site.****
>
> ** **
>
> In regards to the error, please paste in the contents of you vm.args file
> from the target node****
>
> ** **
>
> Thanks****
>
> ** **
>
> Richard****
>
> ** **
>
> On Fri, Feb 22, 2013 at 11:05 PM, <rkevinburton at charter.net> wrote:****
>
> Here is the output from one configuration try:****
>
> ** **
>
> 17:00:08.878 [info] Using http target "172.16.33.138":8098 for worker 2***
> *
>
> ** **
>
> 17:00:08.878 [info] Using pb target "172.16.33.138":8087 for worker 2****
>
> ** **
>
> 17:00:08.993 [debug] Supervisor basho_bench_sup started
> basho_bench_worker:start_link(basho_bench_worker_2, 2) at pid <0.96.0>****
>
> ** **
>
> 17:00:08.999 [info] Using http target "172.16.33.107":8098 for worker 3***
> *
>
> ** **
>
> 17:00:09.001 [info] Using pb target "172.16.33.107":8087 for worker 3****
>
> ** **
>
> 17:00:09.006 [debug] Supervisor basho_bench_sup started
> basho_bench_worker:start_link(basho_bench_worker_3, 3) at pid <0.99.0>****
>
> ** **
>
> 17:00:09.022 [debug] Supervisor net_sup started erl_epmd:start_link() at
> pid <0.104.0>****
>
> ** **
>
> 17:00:09.029 [debug] Supervisor net_sup started auth:start_link() at pid
> <0.105.0>****
>
> ** **
>
> 17:00:09.030 [info] Can't set long node name!****
>
> Please check your configuration****
>
> ** **
>
> 17:00:09.030 [error] Failed to start net_kernel for
> basho_bench_measurement_erlangvm:
> {shutdown,{child,undefined,net_sup_dynamic,{erl_distribution,start_link,[[basho_bench,longnames]]},permanent,1000,supervisor,[erl_distribution]}}
> ****
>
> ** **
>
> There is alot more output but this is the tail end of the output.****
>
> ** **
>
> On Fri, Feb 22, 2013 at 4:34 PM, Richard Shaw wrote:****
>
> ** **
>
>  Hi Kevin, ****
>
> ** **
>
> Always good practise to reply to the mailing list so others can benefit
> from the content ****
>
> ** **
>
> basho bench is the best place to start, If you have a working cluster or
> even a working single node, follow the benchmarking guide[0]  ****
>
> ** **
>
> Please let me see some of the specific errors you're having and we can
> help resolve them. ****
>
> ** **
>
> Outside of basho bench, I recommend benchmarking the raw performance of
> your VM, you can do this using a variety of tools like iostat, iperf or a
> test suite like Phoronix test suite, a very comprehensive FOSS tool[1] look
> at the `aio` test ****
>
> ** **
>
> I'd also recommend benchmarking each time you make a configuration change
> to really understand how effective that change was ****
>
> ** **
>
> [0] http://docs.basho.com/riak/latest/cookbooks/Benchmarking/****
>
> [1] http://www.phoronix-test-suite.com/****
>
> ** **
>
> Kind Regards ****
>
> ** **
>
> Richard ****
>
> ** **
>
> On Fri, Feb 22, 2013 at 10:14 PM, Kevin Burton < *rkevinburton at charter.net
> *> wrote: ****
>
> That was my next question. I have tried to get basho_bench to work but so
> far have been unsuccessful. If you have a simple "hello world" sample
> config I would greatly appreciate it. I have tried a bunch of configs in
> the examples directory but get met with one error or another.  ****
>
> Other than that I have just timed various client driver calls. Mostly
> random reads some writes. ****
>
> ** **
>
> On Feb 22, 2013, at 3:48 PM, Richard Shaw < *richard at basho.com*> wrote: **
> **
>
> ** **
>
> Hi Kevin, ****
>
> What kind of benchmarking have you done on your VMs and what did you use ?
> ****
>
> ** **
>
> Kind Regards ****
>
> ** **
>
> Richard ****
>
> ** **
>
> On Fri, Feb 22, 2013 at 9:49 AM, Kevin Burton < *rkevinburton at charter.net*>
> wrote: ****
>
> Ok I suspected as much. This will give me more ammo when requesting more
> resources. Thank you. ****
>
> Now, back to the original question. Given that the physical hardware is
> taken care of. What parameters are most important when tuning a cluster.
> Again for arguments sake assume the same 4 node cluster with a read-write
> ratio of about 75 percent. ****
>
> ** **
>
> On Feb 21, 2013, at 8:26 PM, Alexander Sicular < *siculars at gmail.com*>
> wrote: ****
>
> ** **
>
> It can't be said enough times but the number one thing you can do to
> ensure that you are getting true performance (not to mention redundancy) is
> to use different physical hardware for each of your nodes. Under no
> circumstances should you have more than one VM (one logical node in a Riak
> cluster) on the same physical hardware. Also, use multiple
> connections/threads/parallelism/whatever on client side and be sure to hit
> all the nodes in the cluster haproxy roundrobin-esque when writing to Riak.
> Everything else is in the noise. ****
>
> ** **
>
> -Alexander Sicular ****
>
> ** **
>
> @siculars ****
>
> ** **
>
> On Feb 21, 2013, at 9:04 PM, Kevin Burton < *rkevinburton at charter.net*>
> wrote: ****
>
> ** **
>
>   ****
>
> There each has about 20-30GB of disk space.  They each are a VM so I am
> not sure how to specify the CPU. They all seem to be 64 bit Intel
> processors but I could tell you the clock speed. The network is 1Gb
> Ethernet. ****
>
>   ****
>
> *From: *  Sean Carey [mailto: *carey@*basho.com]   ****
>
> *Sent: *  Thursday, February 21, 2013 7:59 PM ****
>
> *To: *  Kevin Burton ****
>
> *Cc: *  *riak-users at lists.basho.com*****
>
> *Subject: *  Re: Tuning a Riak cluster. ****
>
>   ****
>
> Kevin, ****
>
> Disk and CPU, and Network? ****
>
>   ****
>
> ** **
>
> Sean Carey ****
>
> @densone ****
>
> On Thursday, February 21, 2013 at 20:31, Kevin Burton wrote: ****
>
>   ****
>
> I have a cluster of 4 machines (4 Linux VM machines each allocated about 1
> Gb of memory – yea I know it isn’t a lot). I would like to get some
> pointers on getting the fastest query time possible given these meager
> resources. Thank you. ****
>
> _______________________________________________ ****
>
> riak-users mailing list ****
>
> *riak-users at lists.basho.com*****
>
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com****
>
>   ****
>
> _______________________________________________ ****
>
> riak-users mailing list ****
>
> *riak-users at lists.basho.com*****
>
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com****
>
> ** **
>
> _______________________________________________ ****
>
> riak-users mailing list ****
>
> *riak-users at lists.basho.com*****
>
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com****
>
>
>
> ****
>
> ** **
>
> ** **
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130301/f9f2f98d/attachment.html>


More information about the riak-users mailing list