Riak search 2 returning multiple stale results for single record

Peter Roberts peter.roberts at piksel.com
Wed Feb 17 07:59:31 EST 2016


Hi Zeeshan,

Configuring the delete_mode to immediate (or a very low millisecond value) does prevent this occurring for me, I hadn’t spotted this setting before.  This is a suitable solution for us at the moment.  There were no Solr errors in the logs so I think it wasn’t being called by Riak.

Are you able to provide me with any more details on how the delete will change in the upcoming releases and when they might be available?

Thanks,
Peter

From: Zeeshan Lakhani <zlakhani at basho.com<mailto:zlakhani at basho.com>>
Date: Monday, 15 February 2016 20:27
To: Peter Roberts <peter.roberts at piksel.com<mailto:peter.roberts at piksel.com>>
Cc: "riak-users at lists.basho.com<mailto:riak-users at lists.basho.com>" <riak-users at lists.basho.com<mailto:riak-users at lists.basho.com>>
Subject: Re: Riak search 2 returning multiple stale results for single record

Hello Peter,

The cleanup for delete, as a call to SOLR, occurs after the reap happens in the backend, depending on delete_mode of course. Did you notice any Solr errors in the logs related to “failed to index” or something similar? Maybe the delete request never hit the solr_core/search_index, and, therefore, has stayed around as a sibling. The logs would help there. Have you set delete_mode to something specific (i.e. http://docs.basho.com/riak/latest/ops/advanced/deletion/#Configuring-Object-Deletion)?

I can tell you that the soon-to-be-released 2.0.7 and 2.2 releases fixes this with more aggressive tactics to handle deletes, especially around datatypes (I see that you’re using maps), instead of waiting for the call after the backend reap/removal.

Looking at previous threads, your W quorum value can also matter - http://riak-users.197444.n3.nabble.com/Deleted-keys-come-back-td4033536.html#a4033576.

For you current situation, an updated re-PUT for those with siblings would resolve them, or via an internal solr cleanup call, or through a reindexing process. If you want to find me on IRC, we can discuss a few others tools we have that may help you get most of the way there until the new release, depending on the questions I asked in the first paragraph.

Thanks.

Zeeshan Lakhani
programmer |
software engineer at @basho

On Feb 15, 2016, at 5:33 AM, Peter Roberts <peter.roberts at piksel.com<mailto:peter.roberts at piksel.com>> wrote:

We’re currently evaluating whether Riak is suitable for our system and have an issue with multiple/stale results being returned from Riak search. We’re reliably seeing this occur when a record is  deleted and a new one created under the same key shortly afterwards – which, based on the _yz_id, causes siblings to be created.

Accessing the record through the map datatype only gives us the expected record.  We’ve considered de-duplicating the result by ID but this could still result in search results where the query matches the old but not the new data. The obsolete records in Riak search/Solr never get cleaned up.

Is this a known issue? Are we missing something on inserting/querying the data? Is it possible to tie the version of data in the Riak search result to the version retrieved from Riak by key?

Below is an example of the Riak search query/result (using Node basho-riak-client) and datatype data. The date.value is set to the creation/update time.

 var searchOptions = {
    indexName: 'minimals_index',
    q: 'name_register:foo',
    presort: ‘score'
}

riakClient.search(searchOptions, function(err, result) {
  console.log('Search result', err, result);
  callback(err, result);
}

Results:
{ numFound: 6,
  maxScore: 0.35434481501579285,
  docs:
   [ { score: 0.35434482,
       _yz_rb: 'minimals',
       _yz_rt: 'maps',
       _yz_rk: 'test:foo',
       _yz_id: '1*maps*minimals*test:foo*13*2EB6xtIHCicwmARKDJ81tS',
       'date_map._type_register': '_date',
       'date_map._value_register': '2016-02-12T11:39:22.942Z',
       name_register: 'foo',
       owner_register: 'test' },
…
     { score: 0.35434482,
       _yz_rb: 'minimals',
       _yz_rt: 'maps',
       _yz_rk: 'test:foo',
       _yz_id: '1*maps*minimals*test:foo*13*4hNPnHzeJpTc7S6nEW8vNb',
       'date_map._type_register': '_date',
       'date_map._value_register': '2016-02-12T13:28:11.785Z',
       name_register: 'foo',
       owner_register: 'test' } ]
}

Riak get by ID:
$ curl -XGET "http://localhost:11098/types/maps/buckets/minimals/datatypes/test:foo"
{"type":"map","value":{"date_map":{"_type_register":"_date","_value_register":"2016-02-12T13:28:11.785Z"},"name_register":"foo","owner_register":"test"},"context":"g2wAAAABaAJtAAAADMHSXwlPAZAxAAAAQmEDag==“}


Thanks,
Peter

_______________________________________________
riak-users mailing list
riak-users at lists.basho.com<mailto:riak-users at lists.basho.com>
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20160217/442e1bef/attachment-0002.html>


More information about the riak-users mailing list