How to best store arbitrarily large Java objects

Russell Brown russell.brown at me.com
Fri Jul 22 04:21:38 EDT 2016


On 22 Jul 2016, at 09:12, Henning Verbeek <hankipanky at gmail.com> wrote:

> Alex,
> thanks for the very quick response.
> 
> On Thu, Jul 21, 2016 at 5:36 PM, Alex Moore <amoore at basho.com> wrote:
>>> I'm beginning to think that I'll need to remodel my data and use CRDTs
>>> for individual fields such as the `TreeMap`. Would that be a better
>>> way?
>> 
>> 
>> This sounds like a plausible idea.  If you do a lot of possibly conflicting
>> updates to the Tree, then a CRDT map would be the way to go.  You could
>> reuse the key from the main object, and just put it in the new
>> buckettype/bucket.
> 
> Looking at the [documentation](http://docs.basho.com/riak/kv/2.1.4/developing/data-types/maps/)
> I assume there are no limits to the amount of entries, right?

There is a size limit as a map is just a riak object like any other. We’re working on decomposed CRDTs, where the Set/Map/etc are split across many keys. We expect Sets are coming soon, Maps are a little further out.

> 
>> If you don't need to update the tree much, you could also just serialize the
>> tree into it's own object - split up the static data and the often updated
>> data, and put them in different buckets that share the same key.
> 
> The tree is built once and read often, rarely appended to. The problem
> with splitting up the object is that the tree makes up about 95% of
> the size, so unless I can split up the tree, it wont help much.

Splitting up CRDTs that are related is probably going to be a problem too, as they need to share some common causal information to merge correctly. See above.

> 
> Thanks again!
> Henning
> 
> PS: It'd be great to have a `Converter` that can be instructed to map
> fields to CRDT through annotations :)

Is there not a java converter that maps an object to a CRDT map already? That would seem like a nice thing to have, though you’d be limited in the types of your fields to sets/registers/booleans/counters/maps it should work nicely.

> 
> -- 
> My other signature is a regular expression.
> http://www.pray4snow.de
> 
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com





More information about the riak-users mailing list