Yokozuna inconsistent search results

Magnus Kessler mkessler at basho.com
Mon Feb 29 03:28:10 EST 2016

On 26 February 2016 at 15:51, Oleksiy Krivoshey <oleksiyk at gmail.com> wrote:

> Yes, AAE is enabled:
> anti_entropy = active
> [...]

> However the output of "riak-admin search aae-status" looks like this:
> http://oleksiy.sirv.com/misc/search-aae.png
Hi Oleksiy,

there are two partitions on the node that haven't seen their AAE tree
rebuilt in a long time. The reason for this is not clear at the moment,
although we have seen this happening when a partition contains data that
for some reason cannot be indexed with the configured Solr schema.

Please run on the 'riak attach' console:

riak_core_util:rpc_every_member_ann(yz_entropy_mgr, expire_trees, [], 5000).

Afterwards exit the console with "Ctrl-G q".

The AAE trees should start to be rebuilt shortly after. With default
settings, on a cluster with ring size 64 the whole process should finish in
about 1 day and any missing but indexable data should appear in all
assigned Solr instances. During this time you may still observe
inconsistent search results due to the way the coverage query is performed.
Keep an eye open for any errors from one of the yz_ modules in the logs
during this time.

Please let us know if the Search AAE trees can be rebuilt successfully and
if this solved your issue.

Kind Regards,


PS: Please consider subscribing to this subscribers only mailing list to
avoid your messages being held in the moderation queue. Instructions on how
to join can be found at

> On Fri, 26 Feb 2016 at 17:13 Fred Dushin <fdushin at basho.com> wrote:
>> I would check the coverage plans that are being used for the different
>> queries, which you can usually see in the headers of the resulting
>> document.  When you run a search query though yokozuna, it will use a
>> coverage plan from riak core to find a minimal set of nodes (and
>> partitions) to query to get a set of results, and the coverage plan may
>> change every few seconds.  You might be hitting nodes that have
>> inconsistencies or are in need of repair.  Do you have AAE enabled?
>> -Fred
>> > On Feb 26, 2016, at 8:36 AM, Oleksiy Krivoshey <oleksiyk at gmail.com>
>> wrote:
>> >
>> > Hi!
>> >
>> > Riak 2.1.3
>> >
>> > Having a stable data set (no documents deleted in months) I'm receiving
>> inconsistent search results with Yokozuna. For example first query can
>> return num_found: 3000 (correct), the same query repeated in next seconds
>> can return 2998, or 2995, then 3000 again. Similar inconsistency happens
>> when trying to receive data in pages (using start/rows options): sometimes
>> I get the same document twice (in different pages), sometimes some
>> documents are missing completely.
>> >
>> > There are no errors or warning in Yokozuna logs. What should I look for
>> in order to debug the problem?
>> >
>> > Thanks!
Magnus Kessler
Client Services Engineer
Basho Technologies Limited

Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20160229/9fef143c/attachment-0002.html>

More information about the riak-users mailing list