configurable prefix for consistent hashing?
nate at root.org
Wed Nov 9 18:49:26 EST 2011
On Nov 9, 2011, at 3:33 PM, Elias Levy wrote:
> On Wed, Nov 9, 2011 at 3:29 PM, Phil Stanhope <stanhope at gmail.com> wrote:
> Tread carefully here ... by forcing localilty ... you will sacrifice high availability by algorithmically creating a bias and a single point of failure in the cluster.
> You don't have to loose high availability, your data is still being replicated, but you can create hot spots. Known your data.
Correct. Partitioning based on SHA-1(DocumentID) is the same situation as doing it based on SHA-1(entire_key), which is how Riak currently works. Even if "entire_key" and "DocumentID" are both just simple counters, it is the same situation.
We would only need worry if the pair BucketName + DocumentID was not unique (say, skewed towards 0 or something). In that case, we'd need to analyze the distribution of DocumentID values to be sure the partition is balanced.
More information about the riak-users