Limiting the number of entries in a set

Alexander Sicular siculars at gmail.com
Sat Jan 31 16:40:03 EST 2015


Shawn,

I just want to reiterate that Redis data types and Riak data types are not
directly comparable. Riak data types are specifically designed to work in a
fully distributed and automatically convergent environment. And they
service that design very well. Redis data types work the way they work
because of the nature of Redis itself - in memory, single threaded data
store.

I know that the engineering team at Basho are continuously evaluating new
data types and new capabilities for inclusion in Riak. Your feedback and
questions directly impact the product road map. Thanks for letting us know
what you're looking to do and how you would like to use Riak.

Best,
Alexander

On Sat, Jan 31, 2015 at 3:01 PM, Alexander Sicular <siculars at gmail.com>
wrote:

> Riak sets (and data types by extension) are good but not great. The are
> gonna get better over time but still be saddled with the difficulties of
> distribution. Redis is able to do what it does because a. It's not
> distributed and b. It's single threaded. I recommend that if you need the
> atomic guarantees of redis data types you go ahead and use them. And then
> periodically flush your redis data to riak for high availability and fault
> tolerant backup. It's not elegant, it has edge case scenarios but it gets
> you a best of both worlds solution.
>
> Best,
> Alexander
>
>
> @siculars
> http://siculars.posthaven.com
>
> Sent from my iRotaryPhone
>
> On Jan 31, 2015, at 13:11, Shawn Debnath <shawn at debnath.net> wrote:
>
> Good morning,
>
>  Wondering if there is a better way to do truncation of a set if entries
> exceed a certain threshold. I am trying to create a canonical timeline for
> every user in our system, set of UUID strings. We want the number of
> entries to be limited to about 1000 per timeline to keep the size of each
> KV object in check inside Riak and also not store unnecessary data. From
> looking around code and docs, there doesn’t seem to be a Redis style LTRIM
> <http://redis.io/commands/ltrim> option available. So is the only way to
> enforce this is to fetch the set, check count, trim it locally and then
> update?
>
>  My fear is that by using CRDTs, a set add wins over remove in the case
> of a conflict. If there were to be 1000 conflicting concurrent updates each
> trying to trim and add one entry to a set that already contains 1000
> entries, I imagine the  final count of the set would be 2000. This can be
> greatly exasperated on a very “friendly” or popular person’s timeline and
> the code never being able to truncate the set back to 1000.
>
>  Any thoughts or suggestions appreciated.
>
>  Thanks,
>
>  Shawn
> *freenode (sde)*
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20150131/fb7704a1/attachment-0002.html>


More information about the riak-users mailing list