Yokozuna Active Anti-Entropy

Ryan Zezeski rzezeski at basho.com
Mon Apr 8 21:25:13 EDT 2013


Elias,

You are correct in that the _yz_ed field (Entropy Data) is the key to AAE.
 Both KV and YZ build hashtrees for each partition in the ring.  Both trees
are indexed in real-time as writes come in.  Occasionally (default of 1
week) trees are rebuilt from scratch.  In the case of Yokozuna a Lucene
TermsEnum is used to iterate the _yz_ed field which contains the riak
bucket/key and the object hash [1].  During a hashtree exchange KV and YZ
compare the same partition.  If differences are found the KV object is read
and then fed back into the indexing process which a) indexes the data and
b) updates the entry in the hashtree.  Riak KV exchanges are a separate
thing.  I.e. there are exchanges that happen between different partitions
in KV, and then there are exchanges that happen between the same partitions
in KV and YZ.  The idea is that KV AAE will make sure the object replicas
agree and YZ AAE will make sure the indexes match the objects.  The idea
being that KV is the authoritative source of the data.

-Z

[1]
https://github.com/basho/yokozuna/blob/master/priv/java/com/basho/yokozuna/handler/EntropyData.java#L93


On Mon, Apr 8, 2013 at 6:29 PM, Elias Levy <fearsome.lucidity at gmail.com>wrote:

> On Mon, Apr 8, 2013 at 3:25 PM, Alexander Sicular <siculars at gmail.com>wrote:
>
>> I would assume that standard KV AAE triggers a re-write via standard
>> mechanisms which would in turn re-index a key's data in solr. But I could
>> be 100% wrong about that....
>>
>
> My understanding is that this is a completely different AAE.  One that
> compares the state of KV and Solr, not between KV nodes.
>
> I think the answer may be the _yz_ed field Yokozuna adds to the indexed
> documents.  It contains entropy data.  Maybe Yokozuna is reading it to
> build the entropy tree.
>
> Elias Levy
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130408/a4d9b8d9/attachment.html>


More information about the riak-users mailing list