Hi,<div><br></div><div>Here the error that get logged into sasl-error.log file.</div><div><br></div><div><div>=ERROR REPORT==== 29-Apr-2011::14:45:37 ===</div><div>** State machine <0.3117.0> terminating </div><div>** Last event in was timeout</div>

<div>** When State == initialize</div><div>**      Data  == {state,<0.3101.0>,plain,<<>>,undefined,undefined,undefined,</div><div>                  undefined,<<"Games">>,</div><div>

                  {<<"Games">>,[]},</div><div>                  60000,69622590,</div><div>                  {chstate,'<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>',</div><div>

                   [{'<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>',{4,63470357906}}],</div><div>                   {64,</div><div>                    [{0,'<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {22835963083295358096932575511191922182123945984,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {45671926166590716193865151022383844364247891968,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {68507889249886074290797726533575766546371837952,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {91343852333181432387730302044767688728495783936,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {114179815416476790484662877555959610910619729920,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {137015778499772148581595453067151533092743675904,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {159851741583067506678528028578343455274867621888,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {182687704666362864775460604089535377456991567872,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {205523667749658222872393179600727299639115513856,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {228359630832953580969325755111919221821239459840,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {251195593916248939066258330623111144003363405824,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {274031556999544297163190906134303066185487351808,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {296867520082839655260123481645494988367611297792,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {319703483166135013357056057156686910549735243776,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {342539446249430371453988632667878832731859189760,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {365375409332725729550921208179070754913983135744,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {388211372416021087647853783690262677096107081728,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {411047335499316445744786359201454599278231027712,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {433883298582611803841718934712646521460354973696,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {456719261665907161938651510223838443642478919680,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {479555224749202520035584085735030365824602865664,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {502391187832497878132516661246222288006726811648,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {525227150915793236229449236757414210188850757632,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {548063113999088594326381812268606132370974703616,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {570899077082383952423314387779798054553098649600,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {593735040165679310520246963290989976735222595584,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {616571003248974668617179538802181898917346541568,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {639406966332270026714112114313373821099470487552,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {662242929415565384811044689824565743281594433536,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {685078892498860742907977265335757665463718379520,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {707914855582156101004909840846949587645842325504,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {730750818665451459101842416358141509827966271488,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {753586781748746817198774991869333432010090217472,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {776422744832042175295707567380525354192214163456,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {799258707915337533392640142891717276374338109440,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {822094670998632891489572718402909198556462055424,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {844930634081928249586505293914101120738586001408,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {867766597165223607683437869425293042920709947392,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {890602560248518965780370444936484965102833893376,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {913438523331814323877303020447676887284957839360,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {936274486415109681974235595958868809467081785344,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {959110449498405040071168171470060731649205731328,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {981946412581700398168100746981252653831329677312,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {1004782375664995756265033322492444576013453623296,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {1027618338748291114361965898003636498195577569280,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {1050454301831586472458898473514828420377701515264,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {1073290264914881830555831049026020342559825461248,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {1096126227998177188652763624537212264741949407232,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {1118962191081472546749696200048404186924073353216,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {1141798154164767904846628775559596109106197299200,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {1164634117248063262943561351070788031288321245184,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {1187470080331358621040493926581979953470445191168,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {1210306043414653979137426502093171875652569137152,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {1233142006497949337234359077604363797834693083136,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {1255977969581244695331291653115555720016817029120,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {1278813932664540053428224228626747642198940975104,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {1301649895747835411525156804137939564381064921088,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {1324485858831130769622089379649131486563188867072,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {1347321821914426127719021955160323408745312813056,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {1370157784997721485815954530671515330927436759040,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div>

<div>                     {1392993748081016843912887106182707253109560705024,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {1415829711164312202009819681693899175291684651008,</div>

<div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                     {1438665674247607560106752257205091097473808596992,</div><div>                      '<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'}]},</div>

<div>                   {dict,4,16,16,8,80,48,</div><div>                    {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},</div><div>                    {{[],[],[],[],</div><div>                      [[{bucket,<<"Games">>}|</div>

<div>                        {meta_entry,</div><div>                         [{name,<<"Games">>},</div><div>                          {n_val,3},</div><div>                          {allow_mult,false},</div>

<div>                          {last_write_wins,false},</div><div>                          {precommit,</div><div>                           [{struct,</div><div>                             [{<<"mod">>,<<"riak_search_kv_hook">>},</div>

<div>                              {<<"fun">>,<<"precommit">>}]}]},</div><div>                          {postcommit,[]},</div><div>                          {chash_keyfun,{riak_core_util,chash_std_keyfun}},</div>

<div>                          {linkfun,</div><div>                           {modfun,riak_kv_wm_link_walker,mapreduce_linkfun}},</div><div>                          {old_vclock,86400},</div><div>                          {young_vclock,20},</div>

<div>                          {big_vclock,50},</div><div>                          {small_vclock,10},</div><div>                          {r,quorum},</div><div>                          {w,quorum},</div><div>                          {dw,quorum},</div>

<div>                          {rw,quorum}],</div><div>                         63470357892}]],</div><div>                      [],[],</div><div>                      [[{bucket,<<"User">>}|</div><div>

                        {meta_entry,</div><div>                         [{name,<<"User">>},</div><div>                          {n_val,3},</div><div>                          {allow_mult,false},</div>

<div>                          {last_write_wins,false},</div><div>                          {precommit,</div><div>                           [{struct,</div><div>                             [{<<"mod">>,<<"riak_search_kv_hook">>},</div>

<div>                              {<<"fun">>,<<"precommit">>}]}]},</div><div>                          {postcommit,[]},</div><div>                          {chash_keyfun,{riak_core_util,chash_std_keyfun}},</div>

<div>                          {linkfun,</div><div>                           {modfun,riak_kv_wm_link_walker,mapreduce_linkfun}},</div><div>                          {old_vclock,86400},</div><div>                          {young_vclock,20},</div>

<div>                          {big_vclock,50},</div><div>                          {small_vclock,10},</div><div>                          {r,quorum},</div><div>                          {w,quorum},</div><div>                          {dw,quorum},</div>

<div>                          {rw,quorum}],</div><div>                         63470357885}],</div><div>                       [{bucket,<<"Game_instances">>}|</div><div>                        {meta_entry,</div>

<div>                         [{name,<<"Game_instances">>},</div><div>                          {n_val,3},</div><div>                          {allow_mult,false},</div><div>                          {last_write_wins,false},</div>

<div>                          {precommit,</div><div>                           [{struct,</div><div>                             [{<<"mod">>,<<"riak_search_kv_hook">>},</div><div>

                              {<<"fun">>,<<"precommit">>}]}]},</div><div>                          {postcommit,[]},</div><div>                          {chash_keyfun,{riak_core_util,chash_std_keyfun}},</div>

<div>                          {linkfun,</div><div>                           {modfun,riak_kv_wm_link_walker,mapreduce_linkfun}},</div><div>                          {old_vclock,86400},</div><div>                          {young_vclock,20},</div>

<div>                          {big_vclock,50},</div><div>                          {small_vclock,10},</div><div>                          {r,quorum},</div><div>                          {w,quorum},</div><div>                          {dw,quorum},</div>

<div>                          {rw,quorum}],</div><div>                         63470357900}]],</div><div>                      [],[],[],[],[],[],[],</div><div>                      [[{bucket,<<"Game_instances_run">>}|</div>

<div>                        {meta_entry,</div><div>                         [{name,<<"Game_instances_run">>},</div><div>                          {n_val,3},</div><div>                          {allow_mult,false},</div>

<div>                          {last_write_wins,false},</div><div>                          {precommit,</div><div>                           [{struct,</div><div>                             [{<<"mod">>,<<"riak_search_kv_hook">>},</div>

<div>                              {<<"fun">>,<<"precommit">>}]}]},</div><div>                          {postcommit,[]},</div><div>                          {chash_keyfun,{riak_core_util,chash_std_keyfun}},</div>

<div>                          {linkfun,</div><div>                           {modfun,riak_kv_wm_link_walker,mapreduce_linkfun}},</div><div>                          {old_vclock,86400},</div><div>                          {young_vclock,20},</div>

<div>                          {big_vclock,50},</div><div>                          {small_vclock,10},</div><div>                          {r,quorum},</div><div>                          {w,quorum},</div><div>                          {dw,quorum},</div>

<div>                          {rw,quorum}],</div><div>                         63470357906}]]}}}},</div><div>                  undefined}</div><div>** Reason for termination = </div><div>** {timeout,{gen_server,call,</div>

<div>                        [{riak_kv_keylister_master,'<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>'},</div><div>                         {start_kl,69622590,<0.3117.0>,{<<"Games">>,[]}}]}}</div>

<div><br></div><div>=CRASH REPORT==== 29-Apr-2011::14:45:38 ===</div><div>  crasher:</div><div>    initial call: riak_kv_keys_fsm:init/1</div><div>    pid: <0.3117.0></div><div>    registered_name: []</div><div>    exception exit: {timeout,{gen_server,call,[{riak_kv_keylister_master,<a href="mailto:riaksearch@127.0.0.1">riaksearch@127.0.0.1</a>},{start_kl,69622590,<0.3117.0>,{<<"Games">>,[]}}]}}</div>

<div>      in function  gen_fsm:terminate/7</div><div>      in call from proc_lib:init_p_do_apply/3</div><div>    ancestors: [<0.3110.0>]</div><div>    messages: []</div><div>    links: []</div><div>    dictionary: []</div>

<div>    trap_exit: true</div><div>    status: running</div><div>    heap_size: 610</div><div>    stack_size: 24</div><div>    reductions: 9906</div><div>  neighbours:</div><div><br></div><div><br></div><div>And the functions that i have made are </div>

<div>%% gets all the keys in the bucket</div><div><div>get_bucket_keys(Bucket) -></div><div>    {ok, Pid} = riakc_pb_socket:start_link("127.0.0.1", 8087),</div><div>    {ok, List} = riakc_pb_socket:list_keys(Pid, Bucket,infinity),</div>

<div>    riakc_pb_socket:stop(Pid),</div><div>    {ok,lists:foldr(fun(Key, Acc) -></div><div><span class="Apple-tab-span" style="white-space:pre">                   </span>    [binary_to_list(Key)|Acc]</div><div><span class="Apple-tab-span" style="white-space:pre">                </span>    end, [], List)}.</div>

</div><div><br></div><div>%% select the Key and get the values to return</div><div><div>select(DB, Key, Values_to_return) -></div><div>    {ok, Pid} = riakc_pb_socket:start_link("127.0.0.1", 8087),</div><div>

    Ret = </div><div><span class="Apple-tab-span" style="white-space:pre">        </span>case riakc_pb_socket:get(Pid, list_to_binary(DB), </div><div><span class="Apple-tab-span" style="white-space:pre">                           </span> list_to_binary(Key)) of </div>

<div><span class="Apple-tab-span" style="white-space:pre">      </span>    {ok, FromDBObject} -> </div><div><span class="Apple-tab-span" style="white-space:pre">                </span>Val = to_list(binary_to_term(riakc_obj:get_value(FromDBObject))),</div>

<div><span class="Apple-tab-span" style="white-space:pre">              </span>riakc_pb_socket:stop(Pid),</div><div><span class="Apple-tab-span" style="white-space:pre">           </span>Return = </div><div><span class="Apple-tab-span" style="white-space:pre">            </span>    lists:foldr(</div>

<div><span class="Apple-tab-span" style="white-space:pre">              </span>      fun(Key_for_return, Acc)-></div><div><span class="Apple-tab-span" style="white-space:pre">                  </span>      case proplists:get_value(Key_for_return, Val) of</div>

<div><span class="Apple-tab-span" style="white-space:pre">                              </span>  undefined -> Acc;</div><div><span class="Apple-tab-span" style="white-space:pre">                               </span>  Sth -> [{Key_for_return,Sth} | Acc]</div><div><span class="Apple-tab-span" style="white-space:pre">                     </span>      end</div>

<div><span class="Apple-tab-span" style="white-space:pre">              </span>      end, [], Values_to_return),</div><div><span class="Apple-tab-span" style="white-space:pre">            </span>{ok, Return};</div><div><span class="Apple-tab-span" style="white-space:pre">        </span>    {error, Msg} -></div>

<div><span class="Apple-tab-span" style="white-space:pre">              </span>{error, Msg}</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>end,    </div><div>    Ret.</div></div><div><br></div><div>And I do the following to get all the keys in the bucket and then find some field from that key and return the value.</div>

<div><div>{ok, Key_list} = db_api:get_bucket_keys("Games"),</div><div>    lists:foldl(</div><div>      fun(Key, Acc) -></div><div><span class="Apple-tab-span" style="white-space:pre">  </span>      case db_api:select("Games", Key, ["game_name"]) of</div>

<div><span class="Apple-tab-span" style="white-space:pre">              </span>  {ok, []} -></div><div><span class="Apple-tab-span" style="white-space:pre">             </span>      Acc;</div><div><span class="Apple-tab-span" style="white-space:pre">           </span>  {ok, List} -> [[{"game_id", Key} | List]|Acc]</div>

<div><span class="Apple-tab-span" style="white-space:pre">      </span>      end</div><div>      end,[], Key_list).</div></div><div><br></div><div><br></div><div>It works for like around 2-3 seconds then it starts to timeout and eventually crash..</div>

<div><br></div><div>//Wilson</div><div><br></div><br><div class="gmail_quote">On Fri, Apr 29, 2011 at 2:01 PM, Rusty Klophaus <span dir="ltr"><<a href="mailto:rusty@basho.com">rusty@basho.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hi Wilson,<div><br></div><div>Do you see any errors or otherwise abnormal looking messages in the logs? That will help diagnose the issue.</div><div><br></div><div>The max requests per second depends very much on your hardware, size of data, and read/write patterns.</div>


<div><br></div><div>Thanks,</div><div>Rusty<br><br><div class="gmail_quote"><div><div></div><div class="h5">On Fri, Apr 29, 2011 at 6:44 AM, Wilson Tuladhar <span dir="ltr"><<a href="mailto:mailwilson007@gmail.com" target="_blank">mailwilson007@gmail.com</a>></span> wrote:<br>


</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div></div><div class="h5">Hi,<div><br></div><div>I am using riaksearch with erlang client and i have made an API that does for the select, insert and other such stuffs. I have built an web application on top of it and everything works fine when i do the manual testing.</div>




<div><br></div><div>But now when i am using the automated load testing through Tsung by sending 100 req/sec, the riak shuts down after few seconds. Does anybody know why this is happening or how much requests can riak handle in a second. I am using only single node currently.</div>




<div><br></div><font color="#888888"><div>//Wilson</div>
</font><br></div></div>_______________________________________________<br>
riak-users mailing list<br>
<a href="mailto:riak-users@lists.basho.com" target="_blank">riak-users@lists.basho.com</a><br>
<a href="http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com" target="_blank">http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br></div>