configurable prefix for consistent hashing?
stanhope at gmail.com
Wed Nov 9 18:29:44 EST 2011
Tread carefully here ... by forcing localilty ... you will sacrifice high
availability by algorithmically creating a bias and a single point of
failure in the cluster.
Cassandra made this easier to do ... and then there were many questions on
the user list about:
"why is my cluster is unbalanced?"
'why is one of my nodes is falling over when the others are not?"
On Thu, Nov 10, 2011 at 4:25 AM, Elias Levy <fearsome.lucidity at gmail.com>wrote:
> On Wed, Nov 9, 2011 at 12:00 PM, <riak-users-request at lists.basho.com>wrote:
>> From: Nate Lawson <nate at root.org>
>> We have been looking into ways to cluster keys to benefit from the
>> LevelDB backend's prefix compression. If we were doing a batch of lookups
>> and the keys from a given document could be grouped on a partition, they
>> could be read with less disk IO. However, consistent hashing in Riak
>> currently spreads keys randomly, based on SHA-1.
>> What I think we could really use is a per-bucket configurable prefix
>> length for consistent hashing. We would then create keys of the form
>> "DocumentID:<key>" and tell Riak to only hash DocumentID to get the
>> partition. This way, keys from the same document would be clustered on the
>> same partition.
>> Is this on your roadmap already? It seems like it wouldn't require too
>> many changes to Riak.
> If you look at the bucket properties by fetching them, you'll notice that
> there is already an entry for specifying a key hashing function. Its not
> documented, but you can read the Erlang code for the existing function
> (pretty much just a call to SHA1), and create your own to do whatever
> partitioning you want.
> riak-users mailing list
> riak-users at lists.basho.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the riak-users