the optimal value of the ring_creation_size

Dmitri Zagidulin dzagidulin at basho.com
Thu Apr 25 14:34:38 EDT 2013


Tom,

Just to emphasize Joe's comment -- 512 should be the _maximum_ you want to
use as your ring size with leveldb/multi backend. But you should probably
use a smaller size, unless your cluster is going to have several dozen
nodes.

The recommended rule of thumb with ring size is "~10 vnodes to a physical
machine". Meaning, if you have a 5 node cluster (and it's going to stay at
5 nodes and not scale that much), using a ring size of 64 is perfectly fine
(since each node is going to have 12 to 13 vnodes on it ( 64 / 5) ).

Say you scale that cluster to 10 nodes, and keep the 64 ring size. Here,
each machine is going to be running 6 to 7 virtual nodes on it (64/10), and
will probably be under-utilized, in terms of hardware resources.
Whereas if you select a ring size of 128 for that 10 node cluster, you're
back to the 12-13 vnodes per machine range.

On the other hand, if you have a 5 node cluster with a ring size of 512,
each machine will be responsible for around 100 vnodes, so unless your
servers are very powerful, they're going to compete with each other for
resources (ram, cpu, etc) and be overwhelmed. Also, keep in mind, that
Secondary Index (2i) queries get slower the bigger your ring size is (since
2i uses "covering" queries -- each request has to contact half of all the
vnodes, so it matters whether it has to contact around 32 vnodes (for ring
size 64) or 256 (for 512)).

So in short, most small to medium sized clusters out there use 128 or 256
ring sizes. If you _know_ you're not going to scale past 7-9 nodes, it's
safe to go with 64. If you know you're going to scale to a giant cluster of
50 nodes or some such, use 512 (though at that point, you should
re-evaluate your use of Secondary Indexes).

Last thought - migrating to a different ring size is a big pain, although
it's doable. (You basically have to resort to logical backup tools --
export all your data from the old cluster to disk, and then import it all
to the new cluster).

Dmitri



On Mon, Apr 22, 2013 at 5:30 PM, Tom Zeng <tom at intridea.com> wrote:

> Ok thanks Joe.  We plan to switch to the Multi backend, so will use 512.
>
>
> On Mon, Apr 22, 2013 at 5:17 PM, Joe Caswell <jcaswell at basho.com> wrote:
>
>> Tom,
>>   There is no hard and fast rule for the "best value."  The optimal value
>> for your situation will need to take into account the physical resources
>> available both in your starting cluster as well as your planned end-state
>> cluster.  If you plan to use secondary indexing, the maximum
>> ring_creation_size you should consider is 512.
>>   There will be a separate concurrent vnode_proxy process for each vnode,
>> and a process for each backend for each vnode.  Each backend will need open
>> file handles and RAM for caching objects.  The backend configuration
>> section of the docs should help plan your backed settings
>> http://docs.basho.com/riak/1.3.1/tutorials/choosing-a-backend/
>>   Your planning must also include failure scenarios.  If any of your
>> nodes crash, the surviving nodes will each start more vnodes to cover the
>> missing node(s).   The 10 vnodes per node recommendation is to ensure that
>> the vnodes from any single failed node can be divided among enough
>> surviving nodes to not leave one node handling significantly more load than
>> the other, but this also is not a hard and fast rule.
>> At this time changing number of partitions in the ring does require a
>> complete rebuild of the cluster,  we do have dynamic ring sizing on the
>> product roadmap, but there is no release date set for that feature.
>>
>> Joe Caswell
>>
>> From: Tom Zeng <tom at intridea.com>
>> Date: Sunday, April 21, 2013 9:38 PM
>> To: <riak-users at lists.basho.com>
>> Subject: the optimal value of the ring_creation_size
>>
>> Hi,
>>
>> I am wondering what's the best value for ring_creation_size, the default
>> is 64. According to the docs 64 will work for a cluster of no more than 6
>> nodes (64 /10), and ring_creation_size of 128 will allow cluster of up to
>> 12 mode.  I am wondering what kind of overhead of is associated with a
>> large ring_creation_size.  Since changing the ring_creation_size will
>> result in rebuilding the cluster(destuctive), would a larger value make
>> more sense and allow scaling by adding more nodes?
>>
>>
>> Thanks,
>> Tom
>>
>> --
>> Tom Zeng
>> Director of Engineering
>> Intridea, Inc. | www.intridea.com
>> tom at intridea.com
>> (o) 888.968.4332 x519
>> (c) 240-643-8728
>>  _______________________________________________ riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130425/3c2edf42/attachment.html>


More information about the riak-users mailing list