Riak ring size

John Kavanagh jkavanagh at zendesk.com
Thu Oct 3 13:59:32 EDT 2013


Thanks , that makes sense and I think it gives me my answer. 

I suppose I'm trying to understand if say , an acceptable perf decrease of ~%5 is a healthy trade off in order to mitigate against a future (but possibly unlikely) capacity cap . 

John 

On 3 Oct 2013, at 18:51, Sean Cribbs <sean at basho.com> wrote:

> Your main concern with greater numbers of partitions is contention for IO (and CPU time for the vnodes to process requests). Consider that with 6 nodes, you will have 170(+/- 1) partitions running on each physical node. That's *at least* 170 files open, all scribbling to and reading from disk.  You might get away with 512, but my experience tells me to be wary of that for small clusters. I think unless you're doing big-web-property-level traffic you won't need to grow your cluster as large as you think, especially with machines that beefy.
> 
> 
> On Thu, Oct 3, 2013 at 12:46 PM, John Kavanagh <jkavanagh at zendesk.com> wrote:
> Hi Sean ,
> 
> What would the perform issues be with setting a ring size to 1024 (instead of 128 or 256) on a 6 node cluster? 
> 
> I can see the possibility of memory overflow (which is not a high concern on a 128G machine ) , but other than that , are there any additional concerns (with Bitcask)  ? 
> 
> I'm trying to understand if there is a significant performance degradation with a value of 1024 versus future proofing growth.
> 
> Cheers,
> 
> John 
> 
> On 3 Oct 2013, at 16:33, Sean Cribbs <sean at basho.com> wrote:
> 
>> Hi John,
>> 
>> Our rule-of-thumb is 8-64 partitions per physical node, so a good starting point for a 6-node cluster would be 128 or 256. 256 will let you expand up to about 30 nodes without needing a ring resize.
>> 
>> 
>> On Thu, Oct 3, 2013 at 10:14 AM, John Kavanagh <john at kavanista.com> wrote:
>> Hi all,
>> 
>> I'm trying to understand the optimal ring size / vnode config for a 6 node Riak cluster, which will most likely expand to many more nodes over time .  Does everyone bump from the default of 64 to 1024 to facilitate future growth ?  I can provide more detailed capacity metrics off list if necessary.
>> 
>> Thanks,
>> 
>> John 
>> 
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> 
>> 
>> 
>> 
>> -- 
>> Sean Cribbs <sean at basho.com>
>> Software Engineer
>> Basho Technologies, Inc.
>> http://basho.com/
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> 
> 
> 
> 
> -- 
> Sean Cribbs <sean at basho.com>
> Software Engineer
> Basho Technologies, Inc.
> http://basho.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20131003/2f995fab/attachment.html>


More information about the riak-users mailing list