<div dir="ltr"><div><div>Riak cluster 7 nodes, 1.3.2 ....<br><br>Sometime riak node crashes and not responde at all.<br><br>On one node:<br><br>2014-06-24 14:29:56 =ERROR REPORT====<br>Error in process <0.8303.2> on node 'riak@de3' with exit value: {badarg,[{erlang,binary_to_term,[<<19047 bytes>>],[]},{hashtree,get_disk_bucket,3,[{file,"src/hashtree.erl"},{line,597}]},{hashtree,'-update_levels/3-fun-1-',3,[{file,"src/hashtree.erl"},{line,539}]},{lists,foldl,3,[{file,"lists.erl"},{line,1197}]},{hashtree...<br>
<br>2014-06-24 14:29:56 =CRASH REPORT====<br>  crasher:<br>    initial call: riak_kv_index_hashtree:init/1<br>    pid: <0.8936.2><br>    registered_name: []<br>    exception exit: {{{badmatch,{error,{db_open,"IO error: lock ./data/anti_entropy/1358739803456073806767488242915919369836374786048/LOCK: already held by process"}}},[{hashtree,new_segment_store,2,[{file,"src/hashtree.erl"},{line,499}]},{hashtree,new,2,[{file,"src/hashtree.erl"},{line,215}]},{riak_kv_index_hashtree,do_new_tree,2,[{file,"src/riak_kv_index_hashtree.erl"},{line,426}]},{lists,foldl,3,[{file,"lists.erl"},{line,1197}]},{riak_kv_index_hashtree,init_trees,2,[{file,"src/riak_kv_index_hashtree.erl"},{line,368}]},{riak_kv_index_hashtree,init,1,[{file,"src/riak_kv_index_hashtree.erl"},{line,225}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]},[{gen_server,init_it,6,[{file,"gen_server.erl"},{line,328}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}<br>
    ancestors: [<0.1665.0>,riak_core_vnode_sup,riak_core_sup,<0.129.0>]<br>    messages: []<br>    links: []<br>    dictionary: []<br>    trap_exit: false<br>    status: running<br>    heap_size: 987<br>    stack_size: 24<br>
    reductions: 604<br>  neighbours:<br>2014-06-24 14:30:50 =ERROR REPORT====<br>Error in process <0.9292.2> on node 'riak@de3' with exit value: {badarg,[{erlang,binary_to_term,[<<19399 bytes>>],[]},{hashtree,get_disk_bucket,3,[{file,"src/hashtree.erl"},{line,597}]},{hashtree,'-update_levels/3-fun-1-',3,[{file,"src/hashtree.erl"},{line,539}]},{lists,foldl,3,[{file,"lists.erl"},{line,1197}]},{hashtree...<br>
<br>2014-06-24 14:30:50 =CRASH REPORT====<br>  crasher:<br>    initial call: riak_kv_index_hashtree:init/1<br>    pid: <0.10819.2><br>    registered_name: []<br>    exception exit: {{{badmatch,{error,{db_open,"IO error: lock ./data/anti_entropy/536645132457440915277915524513010171279912730624/LOCK: already held by process"}}},[{hashtree,new_segment_store,2,[{file,"src/hashtree.erl"},{line,499}]},{hashtree,new,2,[{file,"src/hashtree.erl"},{line,215}]},{riak_kv_index_hashtree,do_new_tree,2,[{file,"src/riak_kv_index_hashtree.erl"},{line,426}]},{lists,foldl,3,[{file,"lists.erl"},{line,1197}]},{riak_kv_index_hashtree,init_trees,2,[{file,"src/riak_kv_index_hashtree.erl"},{line,368}]},{riak_kv_index_hashtree,init,1,[{file,"src/riak_kv_index_hashtree.erl"},{line,225}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]},[{gen_server,init_it,6,[{file,"gen_server.erl"},{line,328}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}<br>
    ancestors: [<0.1140.0>,riak_core_vnode_sup,riak_core_sup,<0.129.0>]<br>    messages: []<br>    links: []<br>    dictionary: []<br>    trap_exit: false<br>    status: running<br>    heap_size: 987<br>    stack_size: 24<br>
    reductions: 601<br>  neighbours:<br><br><br>What does it mean ?<br><br></div>It seems - always same partitions:<br><br>536645132457440915277915524513010171279912730624<br>924856504873462002925769308203272848376019812352<br>
<br></div>Thanks.<br></div>