KV Index Tictac Tree - potential feature proposal for Riak 3.0

Martin Sumner martin.sumner at adaptip.co.uk
Thu Apr 19 12:12:08 EDT 2018


One of the design goals around Leveled was to make the production of
anti-entropy trees easier to achieve.  Originally I intended this as a
backend feature (i.e. it would be only available in Riak + leveled).

Over the past few months I've worked on KV Index Tictac Tree - which is an
AAE library, that takes some of the ideas from the Leveled/AAE work, and
makes them generally available across other backends.

The library is now ready to share as a working prototype:

https://github.com/martinsumner/kv_index_tictactree

The intention is to use this as a building block to provide a drop-in
replacement to existing Riak AAE for Riak 3.0.  Anticipated features are:

- Will run in parallel to existing AAE, so can support a controlled
migration;
- Can rapidly provide Merkle Trees merged over coverage plans - to make
inter-cluster and inter-database replication much faster and easier to
implement;
- Can be run without a dedicated AAE keystore if using the leveled
backend,  but still provide equivalent features when using other backends;
- Supports coverage folds across the AAE store per-bucket for reporting
queries;
- Will be "always on", so no loss of anti-entropy capability during AAE
rebuilds;
- Provides access for Repair functions to the clock differences between
objects, so intelligent repair decisions can be made (e.g. no longer
replicate out of date objects due to hash differences).

Over the next month I want to get this plugged into Riak, and run some
performance tests.

Any feedback appreciated.

Regards

Martin

P.S. I will be talking about this at CodeBEAM Stockholm -
http://www.erlang-factory.com/codebeamsto2018
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20180419/42125ef3/attachment.html>


More information about the riak-users mailing list