<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><span style="font-family: UbuntuMono-RegularForPowerline;" class="">Hello Baskar, Santi.</span><div class="" style="font-family: UbuntuMono-RegularForPowerline;"><div class=""><br class=""></div><div class="">I’d definitely like to know what your AAE status is like, as per my last email. The fix here may just be clearing the AAE trees and rebuilding them.</div><div class=""><br class=""></div><div class="">Yes, index creation is currently async and you have to wait for it to be created before sending data. You can poll to check for the index, similar to this function used in the search java-client tests, <a href="https://github.com/basho/riak-java-client/blob/89778ae5a2274ce8693709bcc4c5ea3558d72971/src/test/java/com/basho/riak/client/core/operations/itest/ITestBase.java#L264" class="">https://github.com/basho/riak-java-client/blob/89778ae5a2274ce8693709bcc4c5ea3558d72971/src/test/java/com/basho/riak/client/core/operations/itest/ITestBase.java#L264</a>. We do something similarly in our erlang tests for search. </div><div class=""><br class=""></div><div class="">I’m not aware of a better way right now. Currently, I’m actually working on making index creation synchronous before returning a response (within a timeout), which could solve your issues.</div></div><div class="" style="font-family: UbuntuMono-RegularForPowerline;"><br class=""></div><div class="" style="font-family: UbuntuMono-RegularForPowerline;">Thanks.</div><div class="" style="font-family: UbuntuMono-RegularForPowerline;"><br class=""></div>
<div><br class=""><blockquote type="cite" class=""><div class="">Begin forwarded message:</div><br class="Apple-interchange-newline"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">Date: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class="">March 5, 2015 at 11:21:59 AM EST<br class=""></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">Subject: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class=""><b class="">Re: Query on Riak Search in a cluster of 3 nodes behind ELB is giving different result everytime</b><br class=""></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">From: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class="">Baskar Srinivasan <<a href="mailto:baskar@veradocs.com" class="">baskar@veradocs.com</a>><br class=""></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">To: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class="">Santi Kumar <<a href="mailto:santi@veradocs.com" class="">santi@veradocs.com</a>><br class=""></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">Cc: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class="">Zeeshan Lakhani <<a href="mailto:zlakhani@basho.com" class="">zlakhani@basho.com</a>><br class=""></span></div><br class=""><div class=""><div dir="ltr" class="">Hello Zeeshan,<div class=""><br class=""></div><div class="">I work with Santi on the same server backend component. Our primary issue seems to be that Index creation and Bucket association with the indices is being done via load balancer. After we do that, we move onto actually creating data. When data is created is getting into Riak fine. However not all data make it to the indices. </div><div class=""><br class=""></div><div class="">It appears that depending when a node gets the Riak data persist call it may not have completed index creation and bucket to index association for that node which in turns means that particular data doesn't get indexed anywhere.</div><div class=""><br class=""></div><div class="">Is it possible to ensure that the call to create indices and associate buckets with indices guarantees that this action has been completed on all nodes of a cluster?</div><div class=""><br class=""></div><div class="">The API we are using are as follows:</div><div class=""><br class=""></div><div class="">1. Index creation</div><div class=""><p class="">YokozunaSchema yschema = getSchema();</p><p class="">YokozunaIndex vdIndex = <span class="">new</span> YokozunaIndex(indexName,yschema.getName());</p><div class="">        <span class="">  </span><br class="webkit-block-placeholder"></div><p class="">        <span class="">   </span>StoreIndex storeIndex =<span class="">new</span> StoreIndex.Builder(vdIndex).build();</p><p class="">        <span class="">       </span>RiakFuture<Void, YokozunaIndex> storeIndexFuture = client.executeAsync(storeIndex);</p><p class="">        <span class=""> </span>storeIndexFuture.await();</p><p class="">         <span class="">if</span>(!storeIndexFuture.isSuccess()){</p><p class="">        <span class="">   </span><span class="">    </span><span class="">throw</span> <span class="">new</span> Exception("....");</p><div class="">















<br class="webkit-block-placeholder"></div><p class="">        <span class="">   </span>}</p><p class=""><br class=""></p><p class="">2. Bucket association</p><p class=""><br class=""></p><p class="">StoreBucketProperties storeBucketProps = <span class="">new</span> StoreBucketProperties.Builder(bucket)</p><p class="">        <span class="">    </span><span class="">    </span><span class="">    </span>.withSearchIndex(indexName)</p><p class="">        <span class="">       </span><span class="">    </span><span class="">    </span>.build();</p><div class="">        <span class="">  </span><br class="webkit-block-placeholder"></div><p class="">        <span class="">   </span>RiakFuture<Void,Namespace> storeBucketPropsFuture = client.executeAsync(storeBucketProps);</p><p class="">        <span class="">  </span><span class="">try</span>{</p><p class=""><span class="">      </span>        <span class="">        </span>storeBucketPropsFuture.await();</p><p class=""><span class="">       </span>        <span class="">        </span><span class="">if</span>(storeBucketPropsFuture.isDone() && storeBucketPropsFuture.isSuccess()){</p><p class=""><span class="">        </span>        <span class="">        </span><span class="">    </span><span class="">logger</span>.info(<span class="">" Associated index "</span>+indexName+<span class="">" With bucket "</span>+bucketName);</p><p class=""><span class="">       </span>        <span class="">        </span>}<span class="">else</span>{</p><p class=""><span class="">    </span>        <span class="">        </span><span class="">    </span><span class="">logger</span>.warn(<span class="">" Unable to associate bucket "</span>+bucketName);</p><p class=""><span class="">     </span>        <span class="">        </span>}</p><p class="">        <span class=""> </span>}<span class="">catch</span>(Exception e){</p><p class="">        <span class="">  </span><span class="">    </span><span class="">logger</span>.warn(<span class="">"Issues with "</span>);</p><div class="">





















<br class="webkit-block-placeholder"></div><p class="">    <span class=""> </span><span class="">    </span>}</p><p class=""><br class=""></p><p class="">Regards,</p><p class="">Baskar</p></div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Thu, Mar 5, 2015 at 8:06 AM, Santi Kumar <span dir="ltr" class=""><<a href="mailto:santi@veradocs.com" target="_blank" class="">santi@veradocs.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr" class="">This issue is happening for *:* query only. It doesn't have lot of objects so we are doing *:* query. Ever time it fetches different number when I switched to list all keys and do a multi get its returning all objects </p><p dir="ltr" class="">Another thing what we do is we create indexes on the fly and wait for index creation by checking Listindex operation. Once it found the index then we are associating the Bucket with index. After that we write some entries into that bucket. </p><p dir="ltr" class="">I fetched the key and tried searching again but it's the same again. So read repair would have happened so that wouldn't be the issue I can check AAE</p><p dir="ltr" class="">Can you suggest any better way for these index creations and bucket association? I suspect that might be the reason<br class="">
Is there any way to check what's the data each shard has? I tried solr query with shards param <br class=""></p><p dir="ltr" class="">Thanks <br class="">
Santi </p>
<div class="gmail_quote">On Mar 5, 2015 8:05 PM, "Zeeshan Lakhani" <<a href="mailto:zlakhani@basho.com" target="_blank" class="">zlakhani@basho.com</a>> wrote:<br type="attribution" class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class="">Hello Santi, <br class=""><br class="">Have you deleted an object in that bucket/index at some point? </div><div class=""><br class=""></div><div class="">Please make sure AAE is running by checking search’s AAE status, `riak-admin search aae-status`, and that data exists in the correct directory, `./data/yz_anti_entropy` (<a href="http://docs.basho.com/riak/latest/ops/advanced/configs/search/" target="_blank" class="">http://docs.basho.com/riak/latest/ops/advanced/configs/search/</a>). </div><div class=""><br class=""></div><div class="">You may just need to perform a read-repair by performing a fetch of the object itself first, before performing search queries again.</div><div class=""><br class=""></div><div class="">Thanks.</div><br class=""><div class="">
<div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class=""><div style="font-family: 'Ubuntu Mono for Powerline'; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class=""><div style="font-family: 'Ubuntu Mono'; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class=""><div style="font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class=""><span style="border-collapse:separate;border-spacing:0px" class=""><div style="word-wrap:break-word" class=""><span style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; border-spacing: 0px;" class=""><div style="word-wrap:break-word" class=""><span style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; border-spacing: 0px;" class=""><div style="word-wrap:break-word" class="">Zeeshan Lakhani<br class="">programmer | </div><div style="word-wrap:break-word" class="">software engineer at @basho | </div><div style="word-wrap:break-word" class="">org. member/founder of @papers_we_love | <a href="http://paperswelove.org/" target="_blank" class="">paperswelove.org</a><br class="">twitter => @zeeshanlakhani</div></span></div></span></div></span></div></div></div></div></div></div></div></div></div></div>
</div>
<br class=""><div class=""><blockquote type="cite" class=""><div class="">On Mar 4, 2015, at 9:06 PM, Santi Kumar <<a href="mailto:santi@veradocs.com" target="_blank" class="">santi@veradocs.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class="">But in our case we didnt have spaces in keys. All our keys are UUID's so I wouldn't suspect that. AAE I haven't verified .</div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Thu, Mar 5, 2015 at 7:16 AM, John O'Brien <span dir="ltr" class=""><<a href="mailto:boardom@gmail.com" target="_blank" class="">boardom@gmail.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="">I'd lean towards AAE issues on Yokozuna... Same problems we were having with our 'spaces-in-keys' issue... Once we cleaned those up, things were great again.</div><div class="gmail_extra"><br class=""><div class="gmail_quote"><div class=""><div class="">On Wed, Mar 4, 2015 at 8:36 PM, Santi Kumar <span dir="ltr" class=""><<a href="mailto:santi@veradocs.com" target="_blank" class="">santi@veradocs.com</a>></span> wrote:<br class=""></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class=""><div class=""><p dir="ltr" class="">Raik 2.0.0</p><div class=""><div class="">
<div class="gmail_quote">On Mar 5, 2015 12:31 AM, "Christopher Meiklejohn" <<a href="mailto:cmeiklejohn@basho.com" target="_blank" class="">cmeiklejohn@basho.com</a>> wrote:<br type="attribution" class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br class="">
> On Mar 4, 2015, at 1:15 PM, Santi Kumar <<a href="mailto:santi@veradocs.com" target="_blank" class="">santi@veradocs.com</a>> wrote:<br class="">
><br class="">
> Hi,<br class="">
> We are running into a strange issue with Riak Search. Our setup is with 3 nodes of Riak (with search enabled) in 3 different  ec2 instances behind ELB. App server talks to the cluster through ELB. We are querying for list of objects through *:* query instead of list bucket keys and every time the result is different. We have only 4 objects in that bucket / index, only few times it's giving all 4 but ofther it returns 2 or 3.<br class="">
><br class="">
> This wasn't happening when we have one instance of Riak. Any insight?<br class="">
<br class="">
Hi Santi,<br class="">
<br class="">
Can you provide information regarding what versions of Riak you are running?<br class="">
<br class="">
- Chris<br class="">
<br class="">
Christopher Meiklejohn<br class="">
Senior Software Engineer<br class="">
Basho Technologies, Inc.<br class="">
<a href="mailto:cmeiklejohn@basho.com" target="_blank" class="">cmeiklejohn@basho.com</a></blockquote></div>
</div></div><br class=""></div></div>_______________________________________________<br class="">
riak-users mailing list<br class="">
<a href="mailto:riak-users@lists.basho.com" target="_blank" class="">riak-users@lists.basho.com</a><br class="">
<a href="http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com" target="_blank" class="">http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com</a><br class="">
<br class=""></blockquote></div><br class=""></div>
</blockquote></div><br class=""></div>
_______________________________________________<br class="">riak-users mailing list<br class=""><a href="mailto:riak-users@lists.basho.com" target="_blank" class="">riak-users@lists.basho.com</a><br class=""><a href="http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com" target="_blank" class="">http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com</a><br class=""></div></blockquote></div><br class=""></div></blockquote></div>
</blockquote></div><br class=""></div>
</div></blockquote></div><br class=""></body></html>