Map reduce timeout

Germain Maurice germain.maurice at linkfluence.net
Tue Oct 5 12:05:06 EDT 2010


  Hello everybody,

I'm trying to execute some mapreduce jobs and i have a problem with timeout.
I'm working with riak_kv_version : <<"0.12.1">> with a single node 
"cluster".

My inputs are "people" bucket which contains more than 200k documents. 
So i have decided to set mapreduce timeout to 600000 milliseconds.
Obviously, it doesn't take care of timeout.
You'll find below the JSON object i sent to Riak/mapred and the Internal 
Server Error less than 60 seconds after. I added the error message i got
in riak console.

=====
 > date; perl riak.pl ; date
mardi 5 octobre 2010, 17:47:58 (UTC+0200)
--- "{\"timeout\":600000,\"query\":[{\"map\":{\"source\":\"function(v) { 
var data = Riak.mapValuesJson(v)[0] ; if ( data.faved_what != undefined 
&& data.faved_what.total != undefined ) { return  [ 
data.faved_what.total ] ; } return []; 
}\",\"language\":\"javascript\",\"arg\":[],\"keep\":false}},{\"reduce\":{\"source\":\"function(values, 
arg) { return [ values.reduce( function (item, total) { return item + 
total*1 ; }, 0 ) ] 
}\",\"language\":\"javascript\",\"arg\":[],\"keep\":true}}],\"inputs\":\"people\"}"
--- 500 Internal Server Error

mardi 5 octobre 2010, 17:48:53 (UTC+0200)
=====

********
=ERROR REPORT==== 5-Oct-2010::17:48:53 ===
Failed reduce: {timeout,
                    {gen_server,call,
                        [<0.143.0>,
                         {dispatch,
                             {<0.143.0>,#Ref<0.0.5.43514>},
                             {{jsanon,
<<"function(values, arg) { return [ values.reduce( function (item, 
total) { return item + total*1 ; }, 0 ) ] }">>},
                              [<<"2607">>,<<"261">>,<<"1753">>,<<"81">>,
<<"18190">>,<<"581">>,<<"811">>,<<"732">>,
<<"1831">>,<<"967">>,<<"2650">>,<<"222">>,
<<"674">>,<<"2136">>,<<"1467">>,<<"362">>,
<<"19965">>,<<"2025">>,<<"348">>,<<"4817">>,
<<"286">>],
                              []}},
                         10000]}}

=ERROR REPORT==== 5-Oct-2010::17:48:53 ===
** State machine <0.24966.6> terminating
** Last event in was {inputs,[<<"286">>]}
** When State == executing
**      Data  == {state,1,riak_kv_reduce_phase,
                      {state,
                          {javascript,
                              {reduce,
                                  {jsanon,
<<"function(values, arg) { return [ values.reduce( function (item, 
total) { return item + total*1 ; }, 0 ) ] }">>},
                                  [],true}},
                          [],[]},
                      true,true,undefined,[],undefined,1,<0.24964.6>,660000}
** Reason for termination =
** {error,failed_reduce}
********

Any idea about this ??

Thank you,
I'll go back to work on my slides for saturday :)


-- 
Germain Maurice
Administrateur Système/Réseau
Tel : +33.(0)1.42.43.54.33

http://www.linkfluence.net





More information about the riak-users mailing list