Using cursorMark with '_yz_rk'

Fred Dushin fdushin at basho.com
Wed Sep 21 11:22:11 EDT 2016


Okay, I probably spoke too soon.

While Solr 4.7 supports cursor marks, we do have an issue in Riak (or Yokozuna) whereby it is actually impractical to use cursor marks for query.  The problem is that while Yokozuna uses coverage plans generate a filter query that will guarantee that we get no replicas in a result set, these coverage plans change every few seconds, in order to ensure we are not constantly querying a subset of the cluster (thus possibly creating hot zones in the cluster, especially for query-heavy work loads).

Theoretically you could change the interval by which these coverage plans are updated (by setting the yokozuna cover_tick configuration setting in advanced.config [1]), which would be okay in a development or test environment, but which would be unsuitable in production.

The solution is to pin a query to a coverage plan, so that subsequent iterations of the query with the next cursor will use the same filter, and hence will give you proper result sets.  We do not currently have this implemented in Yokozuna.

-Fred

[1] https://github.com/basho/yokozuna/blob/2.0.4/src/yz_cover.erl#L285

> On Sep 21, 2016, at 10:40 AM, Guillaume Boddaert <guillaume at lighthouse-analytics.co> wrote:
> 
> I'm very curious of your cursorMark implementation, I'm in deep need of that feature.
> 
> From my experience I wasn't even able to trigger a query with my riak version as it was not yet supported by the Solr bundled with it. But I might missed a point with that.
> 
> I'm using 2.1.2.
> 
> Guillaume
> 
> On 21/09/2016 03:28, Vipin Sharma wrote:
>> Hi all,
>>  
>> In our system we have default implementation of querying the data from riak using “pagination”.
>> For some of the queries, with huge number of resulting records (into the tunes of 10,000+) , it is becoming an issue and hence we wanted to change it to use “cursorMark” as suggested here : https://cwiki.apache.org/confluence/display/solr/Pagination+of+Results <https://cwiki.apache.org/confluence/display/solr/Pagination+of+Results>
>>  
>> While using cursorMark, 
>> -          It asks for unique key in the sort field. We didn’t have a unique key of our own so wanted to use “_yz_rk”  but It gives error mentioned below.
>> -          Query is accepted when sort parameter is changed to use “_yz_id” instead but  It gives redundant / duplicate records. It is probably a known issue as mentioned here <https://docs.basho.com/riak/kv/2.1.4/developing/usage/search/> ( Pagination Warning). Solution recommended is to use { _yz_rt asc, _yz_rb asc, _yz_rk asc } instead but for each of them query is returning the following error :
>>  
>>                 "error":{"msg":"Cursor functionality requires a sort containing a uniqueKey field tie breaker","code":400}
>>  
>> Can somebody please share some suggestions on this.
>>  
>> Thanks
>> Vipin 
>>  
>>  
>>  
>>  
>> 
>> 
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com <mailto:riak-users at lists.basho.com>
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>
> 
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com <mailto:riak-users at lists.basho.com>
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20160921/a48127b4/attachment-0002.html>


More information about the riak-users mailing list