High number of Riak buckets

Alexander Sicular siculars at gmail.com
Fri Sep 30 12:23:37 EDT 2016


Hi Vikram,

Bucket maximums aside, why are you modeling in this fashion? How will you retrieve individual keys if you don't know the time stamp in advance? Do you have a lookup somewhere else? Doable as lookup keys or crdts or other systems. Are you relying on listing all keys in a bucket? Definitely don't do that.  

Yes, there is a better way. Use Riak TS. Create a table with a composite primary key of topic and time. You can then retrieve by topic equality and time range. You can then cache those results in deterministic keys as necessary. 

If you don't already know, Riak TS is basically (there are some notable differences) Riak KV plus the time series data model. Riak TS makes all sorts of time series oriented projects easier than modeling them against KV. Oh, and you can also leverage KV buckets alongside TS (resource limitations not withstanding.)

Would love to hear more,
Alexander 

@siculars
http://siculars.posthaven.com

Sent from my iRotaryPhone

> On Sep 29, 2016, at 19:42, Vikram Lalit <vikramlalit at gmail.com> wrote:
> 
> Hi - I am creating a messaging platform wherein am modeling each topic to serve as a separate bucket. That means there can potentially be millions of buckets, with each message from a user becoming a value on a distinct timestamp key.
> 
> My question is there any downside to modeling my data in such a manner? Or can folks advise a better way of storing the same in Riak?
> 
> Secondly, I would like to modify the default bucket properties (n_val) - I understand that such 'custom' buckets have a higher performance overhead due to the extra load on the gossip protocol. Is there a way the default n_val of newly created buckets be changed so that even if I have the above said high number of buckets, there is no performance degrade? Believe there was such a config allowed in app.config but not sure that file is leveraged any more after riak.conf was introduced.
> 
> Thanks much.
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com




More information about the riak-users mailing list