Query on Riak Search in a cluster of 3 nodes behind ELB is giving different result everytime

Zeeshan Lakhani zlakhani at basho.com
Thu Mar 5 15:41:06 EST 2015


Hello Baskar, Santi,

2-15 minutes is a long while, and we’ve not seen index creation/propagation be so slow. I’d definitely take a closer look at how you’re creating these indexes dynamically on the fly, as index creation is typically a more straightforward admin task.

We’ve added defaults to solrconfig.xml to handle most typical use-cases. You can read more about solrconfig.xml at http://wiki.apache.org/solr/SolrConfigXml#mainIndex_Section <http://wiki.apache.org/solr/SolrConfigXml#mainIndex_Section>. You may want to take another look and optimize/improve your schema design to prevent such issues. You can read more about Solr’s performance factors here -> http://wiki.apache.org/solr/SolrPerformanceFactors <http://wiki.apache.org/solr/SolrPerformanceFactors>. 

Thanks.


Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 5, 2015, at 3:00 PM, Baskar Srinivasan <baskar at veradocs.com> wrote:
> 
> Hello Zeeshan,
> 
> Thanks for the pointer regarding waiting for index creation in each node in the cluster.
> 
> Presently, when the indices get created on one node, it takes a full 2-15 minutes for it to get created on other nodes in the cluster. Following are the timestamps on 3 nodes for a single index:
> 
> #Create index request from our server via load balancer
> 11:16:52.999 [http-bio-8080-exec-3] INFO  c.v.s.u.RiakClientUtil - Created index for bsr-test-fromlocal-1-Access_index
> 
> #1st node, immediate creation (12 secs) once call is issued from our server
> 2015-03-05 19:17:04.135 [info] <0.17388.104>@yz_index:local_create:189 Created index bsr-test-fromlocal-1-Access_index with schema
> 
> #2nd node, takes another 4 minutes for creation request to propagate
> 
> 
> 2015-03-05 19:21:17.879 [info] <0.20606.449>@yz_index:local_create:189 Created index bsr-test-fromlocal-1-Access_index
> 
> #3rd node, takes 15 minutes for creation request to propagate
> 
> 
> 2015-03-05 19:32:32.172 [info] <0.14715.94>@yz_index:local_create:189 Created index bsr-test-fromlocal-1-Access_index
> 
> Is there a solr config we can tune to make the 2nd and 3rd node propagation more immediate in the order of < 60 seconds?
> 
> Thanks,
> 
> Baskar
> 
> 
> On Thu, Mar 5, 2015 at 9:11 AM, Zeeshan Lakhani <zlakhani at basho.com <mailto:zlakhani at basho.com>> wrote:
> Hello Santi, Baskar. Please keep your messages on the user group mailing list, btw. Thanks.
> 
> Here’s an example of our testing harness’s wait_for_index function, https://github.com/basho/yokozuna/blob/develop/riak_test/yz_rt.erl#L420 <https://github.com/basho/yokozuna/blob/develop/riak_test/yz_rt.erl#L420>. We check for the index on each of the nodes, which is an approach you can take. 
> 
> And, as I mentioned, I’m currently working on making Index creation synchronous to make this easier.
> 
> If your logs are not pointing to any errors and being that your bucket, index contains so few objects, I’d delete or mv the search-root/index directory (./data/yz/<<index_name>>) and let AAE resync the data, which should then give you consistent results.
> 
> Thanks.
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20150305/2b3cbf8d/attachment-0002.html>


More information about the riak-users mailing list