ID generation techniques

Sargun Dhillon sargun at sargun.me
Wed Aug 20 21:36:03 EDT 2014


My recommendation would be to use GUIDs. Since they are in hex, it's
more difficult to accidentally make the mistake of turning them into a
number due to the nature of Javascript's type system. They do not rely
on an external system, and it is fairly cheap to generate them. If you
want to them to be k-sorted-ish (no guarantees), prefix the timestamp.
Of course, due to the nature of clocks, this can end poorly.

Since you want to write your code to be idempotent, probably as Riak
is a AP, KV, Dynamo-store, you probably want to rely on client-side
key generation.

A heads up though! Buckets are not free if you use custom properties
on Riak 1.X. Buckets are cheap on Riak 2.0 thanks to the work that
Jordan West has done on the metadata system, but again, not free.

On Wed, Aug 20, 2014 at 3:14 PM, Shailesh Mangal
<shailesh.mangal at getzephyr.com> wrote:
> Thanks Sargun,
>
>
> 1. Keys do get exposed to client. We recently learnt about this limitation
> of javascript and we will convert the keys to String when they get to
> javascript.
> 2. Considering Bitcask as storage choice due to its performance, but we
> are open for levelDB as well. BTW, We already have tenantId (UUID) in the
> bucket name.
> 3. Coming from RDBMS background, we are used to seeing incremental Ids.
> K-sort is not a hard requirement but nice to have.
>
> On 8/20/14, 11:03 AM, "Sargun Dhillon" <sargun at sargun.me> wrote:
>
>>I have questions for your question.
>>
>>1. What are you using your keys for? Do they get passed around in to
>>clients in Javascript? This is important because Javascript only
>>reliably implements IEEE 754 floating point, which is limited to 53
>>bits of precision.
>>
>>2. What backend are you using? In Bitcask, your keyspace is "costly"
>>whereas in LevelDB, keys on disk are "cheap." Additionally, having
>>k-sorted writes helps your story around levelDB compactions?
>>
>>3. Is there any reason you want them K-sorted, other than to make
>>LevelDB compactions a bit nicer?
>>
>>On Wed, Aug 20, 2014 at 10:23 AM, Shailesh Mangal
>><shailesh.mangal at getzephyr.com> wrote:
>>> Hi,
>>>
>>> I wanted to ask what are popular, scalable ID generation techniques for
>>> storing data in RIAK. Some ideas that we are toying with:
>>>
>>> ID Generation server (like Flikr ticket server) - Numeric
>>> Twitter's Snowflake like implementation- Alphanumeric
>>> Riak¹s auto ID generation- Alphanumeric
>>>
>>>
>>> So far, we are thinking of going with option 2 as it doesn¹t require
>>>node
>>> coordination, k-sorted. But we would prefer a uncoordinated numeric
>>>Unique
>>> Ids.
>>>
>>> - Shailesh
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> riak-users at lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>




More information about the riak-users mailing list