Riak Recap for Dec. 13 - 14

Daniel Woo daniel.y.woo at gmail.com
Thu Dec 16 21:01:25 EST 2010


Hi Mark,

Thanks for your explanation, so in this case the partitions would be
re-distributed

*from*
Node1: p1 ~ p16
Node2: p17 ~ p32
Node3: p33 ~ p48
Node4: p49 ~ p64

*to *
Node1: p1 ~ p13 (remove 3 partitions)
Node2: p17 ~ p29 (remove 3 partitions)
Node3: p33 ~ p45 (remove 3 partitions)
Node4: p49 ~ p61 (remove 3 partitions)
Node5: p14, 15, 16, 30, 31, 32, 46, 47, 48, 62, 63, 64 (aprox 1/5 partitions
will be transferred to this new node)

Since there is no centralized node in Riak, how do we know the partition 32
is moved to node 5 from the client caller? Cassandra seems break half of the
adjacent node's data into the new node, that will be easy for the client to
search for the datum just around the node-circle clockwise, although it
causes unbalanced data distribution and you have to move them by command
lines. Riak seems to have this solved by moving partitions into new nodes
equally, that's very interesting, how do you guys make it? If the client
caller queries for partition 32 which was originally on node 2, how do the
client know it's on a new node now?

Thanks,
Daniel

On Fri, Dec 17, 2010 at 6:56 AM, Mark Phillips <mark at basho.com> wrote:

> Hey Daniel,
>
> [snip]
>
> > So, I guess Riak would have to re-hash the whole partitions into all the
> 5
> > nodes, right? Is this done lazily when the node finds the requested data
> is
> > missing?
> > Or is there a way to handle this with consistent re-hashing so we can
> avoid
> > moving data around when new nodes added?
> >
>
> Riak won't rehash all the partitions in the ring when new nodes are
> added. When you go from 4 -> 5 nodes, for example, approx. 1/5 of the
> existing partitions are transferred to the new node. The other 4/5s of
> the partitions will remain unchanged. As far as moving data around
> when new nodes are added, this is impossible to avoid. Data needs to
> be handed off to be spread around the ring.
>
> Hope that helps.
>
> Mark
>



-- 
Thanks & Regards,
Daniel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20101217/52d70a5d/attachment.html>


More information about the riak-users mailing list