How to best store arbitrarily large Java objects
russell.brown at me.com
Fri Jul 22 04:21:38 EDT 2016
On 22 Jul 2016, at 09:12, Henning Verbeek <hankipanky at gmail.com> wrote:
> thanks for the very quick response.
> On Thu, Jul 21, 2016 at 5:36 PM, Alex Moore <amoore at basho.com> wrote:
>>> I'm beginning to think that I'll need to remodel my data and use CRDTs
>>> for individual fields such as the `TreeMap`. Would that be a better
>> This sounds like a plausible idea. If you do a lot of possibly conflicting
>> updates to the Tree, then a CRDT map would be the way to go. You could
>> reuse the key from the main object, and just put it in the new
> Looking at the [documentation](http://docs.basho.com/riak/kv/2.1.4/developing/data-types/maps/)
> I assume there are no limits to the amount of entries, right?
There is a size limit as a map is just a riak object like any other. We’re working on decomposed CRDTs, where the Set/Map/etc are split across many keys. We expect Sets are coming soon, Maps are a little further out.
>> If you don't need to update the tree much, you could also just serialize the
>> tree into it's own object - split up the static data and the often updated
>> data, and put them in different buckets that share the same key.
> The tree is built once and read often, rarely appended to. The problem
> with splitting up the object is that the tree makes up about 95% of
> the size, so unless I can split up the tree, it wont help much.
Splitting up CRDTs that are related is probably going to be a problem too, as they need to share some common causal information to merge correctly. See above.
> Thanks again!
> PS: It'd be great to have a `Converter` that can be instructed to map
> fields to CRDT through annotations :)
Is there not a java converter that maps an object to a CRDT map already? That would seem like a nice thing to have, though you’d be limited in the types of your fields to sets/registers/booleans/counters/maps it should work nicely.
> My other signature is a regular expression.
> riak-users mailing list
> riak-users at lists.basho.com
More information about the riak-users