the optimal value of the ring_creation_size
sean at basho.com
Thu Apr 25 18:37:27 EDT 2013
Minor correction to Dmitri's calcuation: the size of the covering set of
vnodes is RingSize / NVal, or 64 / 3 ~= 22 for the default configuration.
That said, it is still what one would consider "spammy" and should be done
On Thu, Apr 25, 2013 at 11:10 PM, Dev Vasantharajan <dev at sourcebits.com>wrote:
> Speaking of ring resizing, this SO answer from Dmitri could also be good
> reference (if you ever wanna go there).
> On Fri, Apr 26, 2013 at 12:04 AM, Dmitri Zagidulin <dzagidulin at basho.com>wrote:
>> Just to emphasize Joe's comment -- 512 should be the _maximum_ you want
>> to use as your ring size with leveldb/multi backend. But you should
>> probably use a smaller size, unless your cluster is going to have several
>> dozen nodes.
>> The recommended rule of thumb with ring size is "~10 vnodes to a physical
>> machine". Meaning, if you have a 5 node cluster (and it's going to stay at
>> 5 nodes and not scale that much), using a ring size of 64 is perfectly fine
>> (since each node is going to have 12 to 13 vnodes on it ( 64 / 5) ).
>> Say you scale that cluster to 10 nodes, and keep the 64 ring size. Here,
>> each machine is going to be running 6 to 7 virtual nodes on it (64/10), and
>> will probably be under-utilized, in terms of hardware resources.
>> Whereas if you select a ring size of 128 for that 10 node cluster, you're
>> back to the 12-13 vnodes per machine range.
>> On the other hand, if you have a 5 node cluster with a ring size of 512,
>> each machine will be responsible for around 100 vnodes, so unless your
>> servers are very powerful, they're going to compete with each other for
>> resources (ram, cpu, etc) and be overwhelmed. Also, keep in mind, that
>> Secondary Index (2i) queries get slower the bigger your ring size is (since
>> 2i uses "covering" queries -- each request has to contact half of all the
>> vnodes, so it matters whether it has to contact around 32 vnodes (for ring
>> size 64) or 256 (for 512)).
>> So in short, most small to medium sized clusters out there use 128 or 256
>> ring sizes. If you _know_ you're not going to scale past 7-9 nodes, it's
>> safe to go with 64. If you know you're going to scale to a giant cluster of
>> 50 nodes or some such, use 512 (though at that point, you should
>> re-evaluate your use of Secondary Indexes).
>> Last thought - migrating to a different ring size is a big pain, although
>> it's doable. (You basically have to resort to logical backup tools --
>> export all your data from the old cluster to disk, and then import it all
>> to the new cluster).
>> On Mon, Apr 22, 2013 at 5:30 PM, Tom Zeng <tom at intridea.com> wrote:
>>> Ok thanks Joe. We plan to switch to the Multi backend, so will use 512.
>>> On Mon, Apr 22, 2013 at 5:17 PM, Joe Caswell <jcaswell at basho.com> wrote:
>>>> There is no hard and fast rule for the "best value." The optimal
>>>> value for your situation will need to take into account the physical
>>>> resources available both in your starting cluster as well as your planned
>>>> end-state cluster. If you plan to use secondary indexing, the maximum
>>>> ring_creation_size you should consider is 512.
>>>> There will be a separate concurrent vnode_proxy process for each
>>>> vnode, and a process for each backend for each vnode. Each backend will
>>>> need open file handles and RAM for caching objects. The backend
>>>> configuration section of the docs should help plan your backed settings
>>>> Your planning must also include failure scenarios. If any of your
>>>> nodes crash, the surviving nodes will each start more vnodes to cover the
>>>> missing node(s). The 10 vnodes per node recommendation is to ensure that
>>>> the vnodes from any single failed node can be divided among enough
>>>> surviving nodes to not leave one node handling significantly more load than
>>>> the other, but this also is not a hard and fast rule.
>>>> At this time changing number of partitions in the ring does require a
>>>> complete rebuild of the cluster, we do have dynamic ring sizing on the
>>>> product roadmap, but there is no release date set for that feature.
>>>> Joe Caswell
>>>> From: Tom Zeng <tom at intridea.com>
>>>> Date: Sunday, April 21, 2013 9:38 PM
>>>> To: <riak-users at lists.basho.com>
>>>> Subject: the optimal value of the ring_creation_size
>>>> I am wondering what's the best value for ring_creation_size, the
>>>> default is 64. According to the docs 64 will work for a cluster of no more
>>>> than 6 nodes (64 /10), and ring_creation_size of 128 will allow cluster of
>>>> up to 12 mode. I am wondering what kind of overhead of is associated with
>>>> a large ring_creation_size. Since changing the ring_creation_size will
>>>> result in rebuilding the cluster(destuctive), would a larger value make
>>>> more sense and allow scaling by adding more nodes?
>>>> Tom Zeng
>>>> Director of Engineering
>>>> Intridea, Inc. | www.intridea.com
>>>> tom at intridea.com
>>>> (o) 888.968.4332 x519
>>>> (c) 240-643-8728
>>>> _______________________________________________ riak-users mailing
>>>> list riak-users at lists.basho.com
>>> riak-users mailing list
>>> riak-users at lists.basho.com
>> riak-users mailing list
>> riak-users at lists.basho.com
> *Dev Kanchen *|* *Solution Architect
> *Bangalore* | San Francisco | Wroclaw | Connecticut | Atlanta
> M: +91 96111 07106
> GTalk ID: dev at sourcebits.com | Skype ID: dev.kanchen
> Disclaimer:This email and any attachments are sent in strictest confidence
> for the sole use of the addressee and may contain legally privileged,
> confidential, and proprietary data. If you are not the intended recipient,
> please advise the sender by replying promptly to this email and then delete
> and destroy this email and any attachments without any further use, copying
> or forwarding.
> riak-users mailing list
> riak-users at lists.basho.com
Sean Cribbs <sean at basho.com>
Basho Technologies, Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the riak-users