configurable prefix for consistent hashing?

Phil Stanhope stanhope at
Wed Nov 9 18:29:44 EST 2011

Tread carefully here ... by forcing localilty ... you will sacrifice high
availability by algorithmically creating a bias and a single point of
failure in the cluster.

Cassandra made this easier to do ... and then there were many questions on
the user list about:

"why is my cluster is unbalanced?"
'why is one of my nodes is falling over when the others are not?"

On Thu, Nov 10, 2011 at 4:25 AM, Elias Levy <fearsome.lucidity at>wrote:

> On Wed, Nov 9, 2011 at 12:00 PM, <riak-users-request at>wrote:
>> From: Nate Lawson <nate at>
>> We have been looking into ways to cluster keys to benefit from the
>> LevelDB backend's prefix compression. If we were doing a batch of lookups
>> and the keys from a given document could be grouped on a partition, they
>> could be read with less disk IO. However, consistent hashing in Riak
>> currently spreads keys randomly, based on SHA-1.
>> What I think we could really use is a per-bucket configurable prefix
>> length for consistent hashing. We would then create keys of the form
>> "DocumentID:<key>" and tell Riak to only hash DocumentID to get the
>> partition. This way, keys from the same document would be clustered on the
>> same partition.
>> Is this on your roadmap already? It seems like it wouldn't require too
>> many changes to Riak.
> If you look at the bucket properties by fetching them, you'll notice that
> there is already an entry for specifying a key hashing function.  Its not
> documented, but you can read the Erlang code for the existing function
> (pretty much just a call to SHA1), and create your own to do whatever
> partitioning you want.
> _______________________________________________
> riak-users mailing list
> riak-users at
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the riak-users mailing list