Limiting the number of entries in a set

Alexander Sicular siculars at gmail.com
Sat Jan 31 15:01:36 EST 2015


Riak sets (and data types by extension) are good but not great. The are gonna get better over time but still be saddled with the difficulties of distribution. Redis is able to do what it does because a. It's not distributed and b. It's single threaded. I recommend that if you need the atomic guarantees of redis data types you go ahead and use them. And then periodically flush your redis data to riak for high availability and fault tolerant backup. It's not elegant, it has edge case scenarios but it gets you a best of both worlds solution. 

Best,
Alexander


@siculars
http://siculars.posthaven.com

Sent from my iRotaryPhone

> On Jan 31, 2015, at 13:11, Shawn Debnath <shawn at debnath.net> wrote:
> 
> Good morning,
> 
> Wondering if there is a better way to do truncation of a set if entries exceed a certain threshold. I am trying to create a canonical timeline for every user in our system, set of UUID strings. We want the number of entries to be limited to about 1000 per timeline to keep the size of each KV object in check inside Riak and also not store unnecessary data. From looking around code and docs, there doesn’t seem to be a Redis style LTRIM option available. So is the only way to enforce this is to fetch the set, check count, trim it locally and then update?
> 
> My fear is that by using CRDTs, a set add wins over remove in the case of a conflict. If there were to be 1000 conflicting concurrent updates each trying to trim and add one entry to a set that already contains 1000 entries, I imagine the  final count of the set would be 2000. This can be greatly exasperated on a very “friendly” or popular person’s timeline and the code never being able to truncate the set back to 1000.
> 
> Any thoughts or suggestions appreciated.
> 
> Thanks,
> 
> Shawn
> freenode (sde)
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20150131/480b81a3/attachment-0002.html>


More information about the riak-users mailing list