Riak Search Error - Zero Results

Todd Cullen todd at thoughtleadr.com
Tue Oct 11 21:36:58 EDT 2011


The plot thickens, here's an error we're getting out of what appears to be
the search pre-commit hook:

2011-10-11 20:01:12 =SUPERVISOR REPORT====
     Supervisor: {local,riak_kv_put_fsm_sup}
     Context:    child_terminated
     Reason:
{{{badmatch,{error,{{badmatch,{error,{badarg,[{erlang,binary_to_term,[<<131,108,0,0,0,7,104,4,104,3,109,0,0,0,12,114,101,100,100,105,116,95,112,111,115,116,115,109,0,0,0,22,100,97,116,97,95,99,104,105,108,100,114,101,110,95,100,97,116,97,95,117,114,108,109,0,0,0,140,104,116,116,112,58,47,47,119,119,119,46,100,97,105,108,121,109,97,105,108,46,99,111,46,117,107,47,110,101,119,115,47,97,114,116,105,99,108,101,45,50,48,52,55,49,54,56,47,79,99,99,117,112,121,45,87,97,108,108,45,83,116,114,101,101,116,45,0,0,0,182,131,108,0,0,0,1,104,4,104,3,109,0,0,0,12,114,101,100,100,105,116,95,112,111,115,116,115,109,0,0,0,24,100,97,116,97,95,99,104,105,108,100,114,101,110,95,100,97,116,97,95,100,111,119,110,115,109,0,0,0,1,48,109,0,0,0,88,37,50,70,114,37,50,70,80,117,109,112,107,105,110,115,37,50,70,99,111,109,109,101,110,116,115,37,50,70,108,54,104,52,107,37,50,70,102,114,101,101,95,104,117,110,103,101,114,95,103,97,109,101,115,95,109,111,99,107,105,110,103,106,97,121,95,112,117,109,112,107,105,110,95,115,116,101,110,99,105,108,95,112,100,102,37,50,70,110,7,1,161,232,176,105,15,175,4,108,0,0,0,1,104,2,100,0,1,112,107,0,1,0,106,106,0,0,18,0,131,108,0,0,0,25,104,4,104,3,109,0,0,0,12,114,101,100,100,105,116,95,112,111,115,116,115,109,0,0,0,24,100,97,116,97,95,99,104,105,108,100,114,101,110,95,100,97,116,97,95,116,105,116,108,101,109,0,0,0,8,40,82,101,115,116,105,110,103,109,0,0,0,83,37,50,70,114,37,50,70,110,101,117,114,111,37,50,70,99,111,109,109,101,110,116,115,37,50,70,108,54,104,100,48,37,50,70,115,117,112,95,110,101,117,114,111,95,105,95,97,110,105,109,97,116,101,100,95,97,95,100,105,103,105,116,97,108,95,115,104,111,114,116,95,97,98,111,117,116,95,116,104,101,37,50,70,110,7,1,163,167,26,107,15,175,4,108,0,0,0,1,104,2,100,0,1,112,107,0,1,11,106,104,4,104,3,109,0,0,0,12,114,101,100,100,105,116,95,112,111,115,116,115,109,0,0,0,55,100,97,116,97,95,99,104,105,108,100,114,101,110,95,100,97,116,97,95,114,101,112,108,105,101,115,95,100,97,116,97,95,99,104,105,108,100,114,101,110,95,100,97,116,97,95,98,111,100,121,95,104,116,109,108,109,0,0,0,4,73,39,118,101,109,0,0,0,83,37,50,70,114,37,50,70,110,101,117,114,111,37,50,70,99,111,109,109,101,110,116,115,37,50,70,108,54,104,100,48,37,50,70,115,117,112,95,110,101,117,114,111,95,105,95,97,110,105,109,97,116,101,100,95,97,95,100,105,103,105,116,97,108,95,115,104,111,114,116,95,97,98,111,117,116,95,116,104,101,37,50,70,110,7,1,163,167,26,107,15,175,4,108,0,0,0,1,104,2,100,0,1,112,107,0,1,11,106,104,4,104,3,109,0,0,0,12,114,101,100,100,105,116,95,112,111,115,116,115,109,0,0,0,37,100,97,116,97,95,99,104,105,108,100,114,101,110,95,100,97,116,97,95,109,101,100,105,97,95,111,101,109,98,101,100,95,116,105,116,108,101,109,0,0,0,7,78,101,116,119,111,114,107,109,0,0,0,83,37,50,70,114,37,50,70,110,101,117,114,111,37,50,70,99,111,109,109,101,110,116,115,37,50,70,108,54,104,100,48,37,50,70,115,117,112,95,110,101,117,114,111,95,105,95,97,110,105,109,97,116,101,100,95,97,95,100,105,103,105,116,97,108,95,115,104,111,114,116,95,97,98,111,117,116,95,116,104,101,37,50,70,110,7,1,163,167,26,107,15,175,4,108,0,0,0,1,104,2,100,0,1,112,107,0,1,7,106,104,4,104,3,109,0,0,0,12,114,101,100,100,105,116,95,112,111,115,116,115,109,0,0,0,24,100,97,116,97,95,99,104,105,108,100,114,101,110,95,100,97,116,97,95,100,111,119,110,115,109,0,0,0,1,48,109,0,0,0,83,37,50,70,114,37,50,70,110,101,117,114,111,37,50,70,99,111,109,109,101,110,116,115,37,50,70,108,54,104,100,48,37,50,70,115,117,112,95,110,101,117,114,111,95,105,95,97,110,105,109,97,116,101,100,95,97,95,100,105,103,105,116,97,108,95,115,104,111,114,116,95,97,98,111,117,116,95,116,104,101,37,50,70,110,7,1,163,167,26,107,15,175,4,108,0,0,0,1,104,2,100,0,1,112,107,0,6,0,1,2,4,5,6,106,104,4,104,3,109,0,0,0,12,114,101,100,100,105,116,95,112,111,115,116,115,109,0,0,0,26,100,97,116,97,95,99,104,105,108,100,114,101,110,95,100,97,116,97,95,99,114,101,97,116,101,100,109,0,0,0,12,49,51,49,56,50,53,52,57,50,57,46,48,109,0,0,0,83,37,50,70,114,37,50,70,110,101,117,114,111,37,50,70,99,111,109,109,101,110,116,115,37,50,70,108,54,104,100,48,37,50,70,115,117,112,95,110,101,117,114,111,95,105,95,97,110,105,109,97,116,101,100,95,97,95,100,105,103,105,116,97,108,95,115,104,111,114,116,95,97,98,111,117,116,95,116,104,101,37,50,70,110,7,1,163,167,26,107,15,175,4,108,0,0,0,1,104,2,100,0,1,112,107,0,1,5,106,104,4,104,3,109,0,0,0,12,114,101,100,100,105,116,95,112,111,115,116,115,109,0,0,0,28,100,97,116,97,95,99,104,105,108,100,114,101,110,95,100,97,116,97,95,98,111,100,121,95,104,116,109,108,109,0,0,0,4,106,117,115,116,109,0,0,0,83,37,50,70,114,37,50,70,110,101,117,114,111,37,50,70,99,111,109,109,101,110,116,115,37,50,70,108,54,104,100,48,37,50,70,115,117,112,95,110,101,117,114,111,95,105,95,97,110,105,109,97>>]},{mi_buffer,read_value,2},{mi_buffer,open_inner,3},{mi_buffer,new,1},{mi_server,read_buffers,4},{mi_server,read_buf_and_seg,1},{mi_server,init,1},{gen_server,init_it,6}]}}},[{merge_index_backend,start,2},{riak_search_vnode,init,1},{riak_core_vnode,init,1},{gen_fsm,init_it,6},{proc_lib,init_p_do_apply,3}]}}},[{riak_core_vnode_master,get_vnode,2},{riak_core_vnode_master,handle_call,3},{gen_server,handle_msg,5},{proc_lib,init_p_do_apply,3}]},{gen_server,call,[{riak_search_vnode_master,'riak at IP_ADDRESS
'},{'riak_vnode_req_v1',981946412581700398168100746981252653831329677312,{server,undefined,undefined},{'index_v1',[{<<"reddit_posts">>,<<"data_children_data_replies_data_children_data_replies_data_children_data_replies_data_children_data_subreddit">>,<<"Austin">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,[0]}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_replies_data_children_data_replies_data_children_data_replies_data_children_data_body_html">>,<<"this">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,"
"}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_replies_data_children_data_replies_data_children_data_body_html">>,<<"completely">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,[3]}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_replies_data_children_data_replies_data_children_data_body">>,<<"look">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,[16]}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_downs">>,<<"0">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,[1,2]}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_body">>,<<"I'd">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,"%"}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_thumbnail">>,<<"/static/noimage.png">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,[0]}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_replies_data_children_data_replies_data_children_data_replies_data_children_data_name">>,<<"t1_c2q7zwt">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,[0]}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_replies_data_children_data_name">>,<<"t1_c2qac7h">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,[1]}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_replies_data_children_data_body_html">>,<<"Lake">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,[14]}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_body_html">>,<<"class="md"><p>Just">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,"%"}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_body">>,<<"p.m.,">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,[21]}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_replies_data_children_data_replies_data_children_data_replies_data_children_data_body">>,<<"sad">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,[4]}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_replies_data_children_data_replies_data_children_data_body">>,<<"I">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,[1,4,29,36]}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_replies_data_children_data_body_html">>,<<"seems">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,"&"}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_replies_data_children_data_body">>,<<"comes">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,"""}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_replies_data_children_data_body">>,<<"any">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,"6"}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_replies_data_children_data_body">>,<<"why">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,"1"}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_replies_data_children_data_body">>,<<"be">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,"&5"}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_body_html">>,<<"chaps">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,"4"}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_body_html">>,<<"near">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,[23]}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_body">>,<<"down">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,[31]}],1318381272350848},{<<"reddit_posts">>,<<"data_children_data_body">>,<<"by">>,<<"%2Fr%2FAustin%2Fcomments%2Fl6hoj%2Fhelp_find_mark_bosworth%2F">>,[{p,[5]}],1318381272350848}]}},infinity]}}
     Offender:
[{pid,<0.3996.0>},{name,undefined},{mfargs,{riak_kv_put_fsm,start_link,undefined}},{restart_type,temporary},{shutdown,5000},{child_type,worker}]

We have very recursive data structures being stored. Could that be the cause
of the issues?

@Ryan - I'll try your issues ideas next.

On Tue, Oct 11, 2011 at 6:19 AM, Ryan Zezeski <rzezeski at basho.com> wrote:

> Todd,
>
> Just to humor me can you please paste the result of getting the bucket
> props for `my_bucket`.
>
> Did you enable the search flag for all nodes in the cluster?
>
> Did you restart the nodes after flipping the config?
>
> You can check to see if the index is growing via `du -hs <path to riak data
> dir>/merge_index`.  If that number is not going up while indexing then
> something is definitely wrong.
>
> Another sanity check you could try is indexing a local file in another
> bucket like so;
>
> search-cmd index other_bucket <path to txt file>
>
> -Ryan
>
> On Tue, Oct 11, 2011 at 12:54 AM, Todd Cullen <todd at thoughtleadr.com>wrote:
>
>> Hey all,
>>
>>   I've been bashing my head against the wall most of the day trying to
>> figure this out. I can't get search to return me any results even when it
>> should have some positives -> http://pastebin.com/Tu6F2FEG.
>>
>> I've just upgraded my Riak install from v0.14 -> v1.0. I wasn't using Riak
>> Search previously but have just decided to turn it on and try to get it
>> working with the latest release. I actually need to analyze some data
>> sitting in there via search (needless to say I'm very behind on jamming out
>> a report). I tried to track down an answer on IRC earlier today with no luck
>> in finding a result.
>>
>> Here's the steps I've gone through today:
>>
>>    - Flipped the search configuration to true in the app.config file
>>    - Added the precommit hook to the bucket (lets call it "my_bucket" for
>>    the time being) and verified them via cURL
>>    - Setup a script to GET and rePUT all of the KV's in the my_bucket and
>>    have verified this works
>>    - Checked the logs folder for any error/warning messages (crash.log,
>>    error.log, console.log, erlang.log, sasl.log, etc)
>>    - Tested search via the commandline (e.g. search-cmd search my_bucket
>>    attribute_name:term_that_should_be_here)
>>    - Tested search via browser ->
>>    http://my.awesome.site:8098/solr/my_bucket/select?q=attribute_name:term_that_should_be_here
>>
>> I'm not really sure what else I can do to fix the issue. Any suggestions
>> on what else I should check/verify/double check/triple check? Thanks in
>> advance!
>>
>> --
>> Cheers,
>> Todd Cullen
>> CTO & Co-Founder
>> ThoughtLeadr
>>
>> email: todd at thoughtleadr.com
>> phone: 415-254-0787
>> skype: todd.cullen
>>
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>


-- 
Cheers,
Todd Cullen
CTO & Co-Founder
ThoughtLeadr

email: todd at thoughtleadr.com
phone: 415-254-0787
skype: todd.cullen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20111011/fbe15300/attachment.html>


More information about the riak-users mailing list