Basho Bench HTTP Inserts Causing Innostore Crashes

Brian Bickerton bbickerton at omniti.com
Wed Oct 12 11:49:25 EDT 2011


Riak version: 1.0.0
Backend: InnoStore 1.0.3.1
4-node cluster

I've gotten http basho bench to run successfully with the Bitcask and
LevelDB backends, but I'm having trouble with Innostore. I get no throughput
and all errors. The logs show children crashing due to a "bad argument in
call to erlang:size(undefined)". Basic inserts from curl work and I was able
to install rekon, which leads me to believe it's an issue with the bench
testing itself. Rate and concurrency don't appear to matter.

Error log snip:

2011-10-12 15:28:49.060 [error] <0.4751.1> gen_fsm <0.4751.1> in state
active terminated with reason: bad argument in call to
erlang:size(undefined) in innostore:get/2
2011-10-12 15:28:49.061 [error] <0.4751.1> CRASH REPORT Process <0.4751.1>
with 0 neighbours crashed with reason: bad argument in call to
erlang:size(undefined) in innostore:get/2
2011-10-12 15:28:49.061 [error] <0.95.0> Supervisor riak_core_vnode_sup had
child undefined started with {riak_core_vnode,start_link,undefined} at
<0.4751.1> exit with reason bad argument in call to erlang:size(undefined)
in innostore:get/2 in context child_terminated

Crash Log snip:

2011-10-12 15:28:43 =CRASH REPORT====
  crasher:
    initial call: riak_core_vnode:init/1
    pid: <0.4734.1>
    registered_name: []
    exception exit:
{badarg,[{erlang,size,[undefined]},{innostore,get,2},{riak_kv_innostore_backend,get,3},{riak_kv_vnode,do_get_term,3},{riak_kv_vnode,do_get,4},{riak_core_vnode,vnode_command,3},{gen_fsm,handle_msg,7},{proc_lib,init_p_do_apply,3}]}
      in function  gen_fsm:terminate/7
      in call from proc_lib:init_p_do_apply/3
    ancestors: [riak_core_vnode_sup,riak_core_sup,<0.91.0>]
    messages: [{'EXIT',<0.4736.1>,shutdown},{'EXIT',#Port<0.14591>,normal}]
    links: [<0.95.0>]
    dictionary:
[{{innostore,{store,#Port<0.14591>,<<105,0,0,0,0,0,0,0>>,0}},{store,#Port<0.14591>,<<105,0,0,0,0,0,0,0>>,0}}]
    trap_exit: true
    status: running
    heap_size: 610
    stack_size: 24
    reductions: 3941
  neighbours:
2011-10-12 15:28:43 =SUPERVISOR REPORT====
     Supervisor: {local,riak_core_vnode_sup}
     Context:    child_terminated
     Reason:
{badarg,[{erlang,size,[undefined]},{innostore,get,2},{riak_kv_innostore_backend,get,3},{riak_kv_vnode,do_get_term,3},{riak_kv_vnode,do_get,4},{riak_core_vnode,vnode_command,3},{gen_fsm,handle_msg,7},{proc_lib,init_p_do_apply,3}]}
     Offender:
[{pid,<0.4734.1>},{name,undefined},{mfargs,{riak_core_vnode,start_link,undefined}},{restart_type,temporary},{shutdown,300000},{child_type,worker}]

Basho Bench config (worked with Bitcask and LevelDB):

{mode, max}
{duration, 300}.
{concurrent, 1000}.
{driver, basho_bench_driver_http_raw}.
{code_paths, ["deps/ibrowse"]}.
{http_raw_ips, [the, four, node, addresses]}.
{key_generator, {partitioned_sequential_int, 500000000}}.
{value_generator, {fixed_bin, 15}}.
{http_raw_port, 8098}.
{http_raw_path, "/riak/holder"}.
{operations, [{insert, 1}]}.

Any help would be greatly appreciated.

Brian Bickerton
Web Engineer
OmniTI Computer Consulting, Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20111012/3fae1ab2/attachment.html>


More information about the riak-users mailing list