the optimal value of the ring_creation_size

Sean Cribbs sean at basho.com
Thu Apr 25 18:37:27 EDT 2013


Minor correction to Dmitri's calcuation: the size of the covering set of
vnodes is RingSize / NVal, or 64 / 3 ~= 22 for the default configuration.
 That said, it is still what one would consider "spammy" and should be done
with caution.


On Thu, Apr 25, 2013 at 11:10 PM, Dev Vasantharajan <dev at sourcebits.com>wrote:

> Speaking of ring resizing, this SO answer from Dmitri could also be good
> reference (if you ever wanna go there).
>
>
> http://stackoverflow.com/questions/14685236/migrating-riak-data-when-ring-size-changes
>
>
> On Fri, Apr 26, 2013 at 12:04 AM, Dmitri Zagidulin <dzagidulin at basho.com>wrote:
>
>> Tom,
>>
>> Just to emphasize Joe's comment -- 512 should be the _maximum_ you want
>> to use as your ring size with leveldb/multi backend. But you should
>> probably use a smaller size, unless your cluster is going to have several
>> dozen nodes.
>>
>> The recommended rule of thumb with ring size is "~10 vnodes to a physical
>> machine". Meaning, if you have a 5 node cluster (and it's going to stay at
>> 5 nodes and not scale that much), using a ring size of 64 is perfectly fine
>> (since each node is going to have 12 to 13 vnodes on it ( 64 / 5) ).
>>
>> Say you scale that cluster to 10 nodes, and keep the 64 ring size. Here,
>> each machine is going to be running 6 to 7 virtual nodes on it (64/10), and
>> will probably be under-utilized, in terms of hardware resources.
>> Whereas if you select a ring size of 128 for that 10 node cluster, you're
>> back to the 12-13 vnodes per machine range.
>>
>> On the other hand, if you have a 5 node cluster with a ring size of 512,
>> each machine will be responsible for around 100 vnodes, so unless your
>> servers are very powerful, they're going to compete with each other for
>> resources (ram, cpu, etc) and be overwhelmed. Also, keep in mind, that
>> Secondary Index (2i) queries get slower the bigger your ring size is (since
>> 2i uses "covering" queries -- each request has to contact half of all the
>> vnodes, so it matters whether it has to contact around 32 vnodes (for ring
>> size 64) or 256 (for 512)).
>>
>> So in short, most small to medium sized clusters out there use 128 or 256
>> ring sizes. If you _know_ you're not going to scale past 7-9 nodes, it's
>> safe to go with 64. If you know you're going to scale to a giant cluster of
>> 50 nodes or some such, use 512 (though at that point, you should
>> re-evaluate your use of Secondary Indexes).
>>
>> Last thought - migrating to a different ring size is a big pain, although
>> it's doable. (You basically have to resort to logical backup tools --
>> export all your data from the old cluster to disk, and then import it all
>> to the new cluster).
>>
>> Dmitri
>>
>>
>>
>> On Mon, Apr 22, 2013 at 5:30 PM, Tom Zeng <tom at intridea.com> wrote:
>>
>>> Ok thanks Joe.  We plan to switch to the Multi backend, so will use 512.
>>>
>>>
>>> On Mon, Apr 22, 2013 at 5:17 PM, Joe Caswell <jcaswell at basho.com> wrote:
>>>
>>>> Tom,
>>>>   There is no hard and fast rule for the "best value."  The optimal
>>>> value for your situation will need to take into account the physical
>>>> resources available both in your starting cluster as well as your planned
>>>> end-state cluster.  If you plan to use secondary indexing, the maximum
>>>> ring_creation_size you should consider is 512.
>>>>   There will be a separate concurrent vnode_proxy process for each
>>>> vnode, and a process for each backend for each vnode.  Each backend will
>>>> need open file handles and RAM for caching objects.  The backend
>>>> configuration section of the docs should help plan your backed settings
>>>> http://docs.basho.com/riak/1.3.1/tutorials/choosing-a-backend/
>>>>   Your planning must also include failure scenarios.  If any of your
>>>> nodes crash, the surviving nodes will each start more vnodes to cover the
>>>> missing node(s).   The 10 vnodes per node recommendation is to ensure that
>>>> the vnodes from any single failed node can be divided among enough
>>>> surviving nodes to not leave one node handling significantly more load than
>>>> the other, but this also is not a hard and fast rule.
>>>> At this time changing number of partitions in the ring does require a
>>>> complete rebuild of the cluster,  we do have dynamic ring sizing on the
>>>> product roadmap, but there is no release date set for that feature.
>>>>
>>>> Joe Caswell
>>>>
>>>> From: Tom Zeng <tom at intridea.com>
>>>> Date: Sunday, April 21, 2013 9:38 PM
>>>> To: <riak-users at lists.basho.com>
>>>> Subject: the optimal value of the ring_creation_size
>>>>
>>>> Hi,
>>>>
>>>> I am wondering what's the best value for ring_creation_size, the
>>>> default is 64. According to the docs 64 will work for a cluster of no more
>>>> than 6 nodes (64 /10), and ring_creation_size of 128 will allow cluster of
>>>> up to 12 mode.  I am wondering what kind of overhead of is associated with
>>>> a large ring_creation_size.  Since changing the ring_creation_size will
>>>> result in rebuilding the cluster(destuctive), would a larger value make
>>>> more sense and allow scaling by adding more nodes?
>>>>
>>>>
>>>> Thanks,
>>>> Tom
>>>>
>>>> --
>>>> Tom Zeng
>>>> Director of Engineering
>>>> Intridea, Inc. | www.intridea.com
>>>> tom at intridea.com
>>>> (o) 888.968.4332 x519
>>>> (c) 240-643-8728
>>>>  _______________________________________________ riak-users mailing
>>>> list riak-users at lists.basho.com
>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>
>>>
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> riak-users at lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>>
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>
>
> --
> *Dev Kanchen *|*  *Solution Architect
> *Sourcebits*
>
> *Bangalore* | San Francisco | Wroclaw | Connecticut | Atlanta
> M: +91 96111 07106
> GTalk ID: dev at sourcebits.com | Skype ID: dev.kanchen
>
> Disclaimer:This email and any attachments are sent in strictest confidence
> for the sole use of the addressee and may contain legally privileged,
> confidential, and proprietary data. If you are not the intended recipient,
> please advise the sender by replying promptly to this email and then delete
> and destroy this email and any attachments without any further use, copying
> or forwarding.
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>


-- 
Sean Cribbs <sean at basho.com>
Software Engineer
Basho Technologies, Inc.
http://basho.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130426/f03d1536/attachment.html>


More information about the riak-users mailing list