configurable prefix for consistent hashing?

Elias Levy fearsome.lucidity at gmail.com
Wed Nov 9 15:25:36 EST 2011


On Wed, Nov 9, 2011 at 12:00 PM, <riak-users-request at lists.basho.com> wrote:

> From: Nate Lawson <nate at root.org>
>
> We have been looking into ways to cluster keys to benefit from the LevelDB
> backend's prefix compression. If we were doing a batch of lookups and the
> keys from a given document could be grouped on a partition, they could be
> read with less disk IO. However, consistent hashing in Riak currently
> spreads keys randomly, based on SHA-1.
>
> What I think we could really use is a per-bucket configurable prefix
> length for consistent hashing. We would then create keys of the form
> "DocumentID:<key>" and tell Riak to only hash DocumentID to get the
> partition. This way, keys from the same document would be clustered on the
> same partition.
>
> Is this on your roadmap already? It seems like it wouldn't require too
> many changes to Riak.
>

If you look at the bucket properties by fetching them, you'll notice that
there is already an entry for specifying a key hashing function.  Its not
documented, but you can read the Erlang code for the existing function
(pretty much just a call to SHA1), and create your own to do whatever
partitioning you want.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20111109/a947a742/attachment.html>


More information about the riak-users mailing list