How to call mapreduce from Riak client?

Miroslav Urbanek miroslav.urbanek at gmail.com
Wed Feb 27 10:28:18 EST 2013


Answering my own e-mail:

The function has to be executed on a Riak node. The script should call

rpc:call(Host, riak_kv_mrc_pipe, mapred, [Inputs, Phases])

instead of

riak_kv_mrc_pipe:mapred(Inputs, Phases).

Miro

On Tue, Feb 26, 2013 at 10:39 PM, Miroslav Urbanek
<miroslav.urbanek at gmail.com> wrote:
> Dear Riak users,
>
> I have an Erlang script for counting the number of objects in a Riak bucket:
>
> ...
> Inputs = <<"buckets">>,
> Phases = [{reduce,{modfun,riak_kv_mapreduce,reduce_count_inputs},none,true}],
> ...
> {ok, Client} = riak:client_connect(Host)
> Client:mapred(Inputs, Phases)
> ...
>
> After the update to 1.3, this no longer works. In the mailing list
> archive I found an alternative method:
>
> riak_kv_mrc_pipe:mapred(Inputs, Phases)
>
> However, I'm getting the following error:
>
> escript: exception exit: {noproc,{gen_server,call,
>                                     [riak_kv_mrc_sink_sup,
>                                      {start_child,[<0.2.0>,[]]},
>                                      infinity]}}
>   in function  gen_server:call/3 (gen_server.erl, line 188)
>   in call from riak_kv_mrc_pipe:mapred_stream_sink/3
> (src/riak_kv_mrc_pipe.erl, line 266)
>   in call from riak_kv_mrc_pipe:mapred/3 (src/riak_kv_mrc_pipe.erl, line 219)
>
> What does it mean? What's the optimal way to count number of keys in a
> bucket in Riak 1.3 anyway?
>
> Here's the complete script:
>
> #!/usr/bin/env escript
>
> main(_) ->
>
>     Host = 'riak at domain',
>     Inputs = <<"bucket">>,
>     Phases = [{reduce,{modfun,riak_kv_mapreduce,reduce_count_inputs},none,true}],
>
>     ok = application:start(inets),
>     {ok, _} = net_kernel:start([count]),
>     true = auth:set_cookie('riak'),
>     true = net_kernel:connect_node(Host),
>
>     case riak_kv_mrc_pipe:mapred(Inputs, Phases) of
>         {ok, Result} -> io:format("OK: ~p~n", Result);
>         {error, Reason} -> io:format("Error: ~p~n", Reason)
>     end.
>
> Thanks,
> Miro




More information about the riak-users mailing list