configurable prefix for consistent hashing?

Phil Stanhope stanhope at gmail.com
Wed Nov 9 18:29:44 EST 2011


Tread carefully here ... by forcing localilty ... you will sacrifice high
availability by algorithmically creating a bias and a single point of
failure in the cluster.

Cassandra made this easier to do ... and then there were many questions on
the user list about:

"why is my cluster is unbalanced?"
'why is one of my nodes is falling over when the others are not?"
etc...

On Thu, Nov 10, 2011 at 4:25 AM, Elias Levy <fearsome.lucidity at gmail.com>wrote:

> On Wed, Nov 9, 2011 at 12:00 PM, <riak-users-request at lists.basho.com>wrote:
>
>> From: Nate Lawson <nate at root.org>
>>
>>
>> We have been looking into ways to cluster keys to benefit from the
>> LevelDB backend's prefix compression. If we were doing a batch of lookups
>> and the keys from a given document could be grouped on a partition, they
>> could be read with less disk IO. However, consistent hashing in Riak
>> currently spreads keys randomly, based on SHA-1.
>>
>> What I think we could really use is a per-bucket configurable prefix
>> length for consistent hashing. We would then create keys of the form
>> "DocumentID:<key>" and tell Riak to only hash DocumentID to get the
>> partition. This way, keys from the same document would be clustered on the
>> same partition.
>>
>> Is this on your roadmap already? It seems like it wouldn't require too
>> many changes to Riak.
>>
>
> If you look at the bucket properties by fetching them, you'll notice that
> there is already an entry for specifying a key hashing function.  Its not
> documented, but you can read the Erlang code for the existing function
> (pretty much just a call to SHA1), and create your own to do whatever
> partitioning you want.
>
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20111110/f1ec2ab4/attachment.html>


More information about the riak-users mailing list