the optimal value of the ring_creation_size

Tom Zeng tom at intridea.com
Thu Apr 25 20:39:32 EDT 2013


Oh ok so there is the "spammy" factor to think about with large ring size,
thanks all for the helpful info, will go with smaller ring size (will also
do a bit of benchmarking to see if there's any difference).


On Thu, Apr 25, 2013 at 6:37 PM, Sean Cribbs <sean at basho.com> wrote:

> Minor correction to Dmitri's calcuation: the size of the covering set of
> vnodes is RingSize / NVal, or 64 / 3 ~= 22 for the default configuration.
>  That said, it is still what one would consider "spammy" and should be done
> with caution.
>
>
> On Thu, Apr 25, 2013 at 11:10 PM, Dev Vasantharajan <dev at sourcebits.com>wrote:
>
>> Speaking of ring resizing, this SO answer from Dmitri could also be good
>> reference (if you ever wanna go there).
>>
>>
>> http://stackoverflow.com/questions/14685236/migrating-riak-data-when-ring-size-changes
>>
>>
>> On Fri, Apr 26, 2013 at 12:04 AM, Dmitri Zagidulin <dzagidulin at basho.com>wrote:
>>
>>> Tom,
>>>
>>> Just to emphasize Joe's comment -- 512 should be the _maximum_ you want
>>> to use as your ring size with leveldb/multi backend. But you should
>>> probably use a smaller size, unless your cluster is going to have several
>>> dozen nodes.
>>>
>>> The recommended rule of thumb with ring size is "~10 vnodes to a
>>> physical machine". Meaning, if you have a 5 node cluster (and it's going to
>>> stay at 5 nodes and not scale that much), using a ring size of 64 is
>>> perfectly fine (since each node is going to have 12 to 13 vnodes on it ( 64
>>> / 5) ).
>>>
>>> Say you scale that cluster to 10 nodes, and keep the 64 ring size. Here,
>>> each machine is going to be running 6 to 7 virtual nodes on it (64/10), and
>>> will probably be under-utilized, in terms of hardware resources.
>>> Whereas if you select a ring size of 128 for that 10 node cluster,
>>> you're back to the 12-13 vnodes per machine range.
>>>
>>> On the other hand, if you have a 5 node cluster with a ring size of 512,
>>> each machine will be responsible for around 100 vnodes, so unless your
>>> servers are very powerful, they're going to compete with each other for
>>> resources (ram, cpu, etc) and be overwhelmed. Also, keep in mind, that
>>> Secondary Index (2i) queries get slower the bigger your ring size is (since
>>> 2i uses "covering" queries -- each request has to contact half of all the
>>> vnodes, so it matters whether it has to contact around 32 vnodes (for ring
>>> size 64) or 256 (for 512)).
>>>
>>> So in short, most small to medium sized clusters out there use 128 or
>>> 256 ring sizes. If you _know_ you're not going to scale past 7-9 nodes,
>>> it's safe to go with 64. If you know you're going to scale to a giant
>>> cluster of 50 nodes or some such, use 512 (though at that point, you should
>>> re-evaluate your use of Secondary Indexes).
>>>
>>> Last thought - migrating to a different ring size is a big pain,
>>> although it's doable. (You basically have to resort to logical backup tools
>>> -- export all your data from the old cluster to disk, and then import it
>>> all to the new cluster).
>>>
>>> Dmitri
>>>
>>>
>>>
>>> On Mon, Apr 22, 2013 at 5:30 PM, Tom Zeng <tom at intridea.com> wrote:
>>>
>>>> Ok thanks Joe.  We plan to switch to the Multi backend, so will use 512.
>>>>
>>>>
>>>> On Mon, Apr 22, 2013 at 5:17 PM, Joe Caswell <jcaswell at basho.com>wrote:
>>>>
>>>>> Tom,
>>>>>   There is no hard and fast rule for the "best value."  The optimal
>>>>> value for your situation will need to take into account the physical
>>>>> resources available both in your starting cluster as well as your planned
>>>>> end-state cluster.  If you plan to use secondary indexing, the maximum
>>>>> ring_creation_size you should consider is 512.
>>>>>   There will be a separate concurrent vnode_proxy process for each
>>>>> vnode, and a process for each backend for each vnode.  Each backend will
>>>>> need open file handles and RAM for caching objects.  The backend
>>>>> configuration section of the docs should help plan your backed settings
>>>>> http://docs.basho.com/riak/1.3.1/tutorials/choosing-a-backend/
>>>>>   Your planning must also include failure scenarios.  If any of your
>>>>> nodes crash, the surviving nodes will each start more vnodes to cover the
>>>>> missing node(s).   The 10 vnodes per node recommendation is to ensure that
>>>>> the vnodes from any single failed node can be divided among enough
>>>>> surviving nodes to not leave one node handling significantly more load than
>>>>> the other, but this also is not a hard and fast rule.
>>>>> At this time changing number of partitions in the ring does require a
>>>>> complete rebuild of the cluster,  we do have dynamic ring sizing on the
>>>>> product roadmap, but there is no release date set for that feature.
>>>>>
>>>>> Joe Caswell
>>>>>
>>>>> From: Tom Zeng <tom at intridea.com>
>>>>> Date: Sunday, April 21, 2013 9:38 PM
>>>>> To: <riak-users at lists.basho.com>
>>>>> Subject: the optimal value of the ring_creation_size
>>>>>
>>>>> Hi,
>>>>>
>>>>> I am wondering what's the best value for ring_creation_size, the
>>>>> default is 64. According to the docs 64 will work for a cluster of no more
>>>>> than 6 nodes (64 /10), and ring_creation_size of 128 will allow cluster of
>>>>> up to 12 mode.  I am wondering what kind of overhead of is associated with
>>>>> a large ring_creation_size.  Since changing the ring_creation_size will
>>>>> result in rebuilding the cluster(destuctive), would a larger value make
>>>>> more sense and allow scaling by adding more nodes?
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Tom
>>>>>
>>>>> --
>>>>> Tom Zeng
>>>>> Director of Engineering
>>>>> Intridea, Inc. | www.intridea.com
>>>>> tom at intridea.com
>>>>> (o) 888.968.4332 x519
>>>>> (c) 240-643-8728
>>>>>  _______________________________________________ riak-users mailing
>>>>> list riak-users at lists.basho.com
>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> riak-users mailing list
>>>> riak-users at lists.basho.com
>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>
>>>>
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> riak-users at lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>>
>>
>>
>> --
>> *Dev Kanchen *|*  *Solution Architect
>> *Sourcebits*
>>
>> *Bangalore* | San Francisco | Wroclaw | Connecticut | Atlanta
>> M: +91 96111 07106
>> GTalk ID: dev at sourcebits.com | Skype ID: dev.kanchen
>>
>> Disclaimer:This email and any attachments are sent in strictest
>> confidence for the sole use of the addressee and may contain legally
>> privileged, confidential, and proprietary data. If you are not the intended
>> recipient, please advise the sender by replying promptly to this email and
>> then delete and destroy this email and any attachments without any further
>> use, copying or forwarding.
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>
>
> --
> Sean Cribbs <sean at basho.com>
> Software Engineer
> Basho Technologies, Inc.
> http://basho.com/
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>


-- 
Tom Zeng
Director of Engineering
Intridea, Inc. | www.intridea.com
tom at intridea.com
(o) 888.968.4332 x519
(c) 240-643-8728
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130425/e64b763f/attachment.html>


More information about the riak-users mailing list