Riak 2.0 Search Issues

Zeeshan Lakhani zlakhani at basho.com
Wed Mar 4 14:20:47 EST 2015


Hey Steve,

Sorry to see you’re having new issues.

We’ll have the fix for “space in the key” out soon; it’s currently under review. And, I know that this issue is unrelated.

I have a few different routes/thoughts for you, but are you seeing anything related to `yz_events,handle_info` in your crash logs? Also, can you gist/pastebin me your solr logs? I’d like to seem if it correlates with something we’re currently looking at. 

Thanks. 

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 4, 2015, at 10:39 AM, Steve Garon <steve.garon at gmail.com> wrote:
> 
> Hey all, 
> 
> We were having the "space in the key" bug in our cluster so we went through the whole dataset, backing it up to json file and removing the spaces in the keys. Then we trashed our whole cluster and restart from scratch reimporting the whole data. Everything worked like a charm for two weeks but this weekend, not sure what happened but AAE died again. 
> 
> I have two issues now: 
> 1. AAE is trying to recreate an index that already exists and crashes with an "Already exists" error ... I get this in my error log every 15 seconds
> 2. AAE crashes while iterating through entropy data with a request timeout error every hour followed by tons of failed to index objects with request timeout as well. Stack trace looks like this:
> 
> [error] emulator Error in process <025931.7166> on node 'riak at IP' with exit value: {function_clause,[{yz_entropy,iterate_entropy_data,[<<11 bytes>>,[{continuation,<<159 bytes>>},{limit,100},{partition,12}],#Fun<yz_index_hashtree.5.46188917,{error,{error,req_timeout}}],[{file,"src/yz_entropy.erl"},{line,44}]},{yz_index_hashtree,'-fold_keys/2-lc$^0/1-0-',3,[{file... (TRUNCATED)
> 
> [error] <0.1371.0>@yz_kv:index:215 failed to index object {{<<"TYPE">>,<<"BUCKET">>},<<"KEY">>} with error {"Failed to index docs",{error,req_timeout}} because ... (REPEATED MULTIPLE TIME FOR DIFFERENT KEYS)
> 
> I tried clearing the yz anti-entropy tree and reinitialising the yz_entropy_mgr with no luck. Anything I can do to fix this? 
> 
> Oh FYI, I cannot insert data with spaces in the key anymore because we are using a wrapper on top of riak that will prevent us to do so therefore my issues are not related to this for sure.
> 
> These are some config changes that may be good to know for more context.
> We added this to ibrowse.conf:
> {dest, "localhost", 8093, 100, 1000, []}.
> 
> Jetty is set with minthread 80, acceptors 80, and is using the NIO connector.
> 
> All our solr buckets have filterCache disabled with softcommits set to 10s instead of 1.
> 
> Our riak.conf has background_manager turned on with AAE and handoff using it.
> 
> Thanks,
> 
> Steve
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20150304/c7cc8cdb/attachment-0002.html>


More information about the riak-users mailing list