Map Reduce and long queries -

Bryan Fink bryan at basho.com
Mon Oct 15 10:42:49 EDT 2012


On Mon, Oct 15, 2012 at 4:13 AM, Olav Frengstad <olav at fwt.no> wrote:
> Just as a note, using the Erlang pb client you can use the key filters
> for 2i queries if you include the riak_kv_mapred_filters module in
> your client code path.
…
> 2> Index = {index, <<"test">>, <<"$key">>, <<0>>, <<255>>},
> 2> {ok, Filter} = riak_kv_mapred_filters:build_filter([[<<"ends_with">>,"1"]]),
> 2> MR = [
> 2>   { reduce
> 2>   , {qfun, fun(X, F) -> lists:filter(fun({A, B}) -> F(B) end, X) end}
> 2>   , riak_kv_mapred_filters:compose(Filter)
> 2>   , true}],
> 2> riakc_pb_socket:mapred(Pid, Index, MR).

It's a bit unfortunate that qfun is so useful, because it's also so
fragile. If your client node is not using exactly the same module
version as every node in your Riak cluster, this will fail with
undefined function errors. Absolutely, go ahead and use qfun to learn
with, but keep an eye out for that kind of surprise. Avoid qfun in
production if you can, or your upgrade process will become more
complex and/or prone to failure.

Yes, http://docs.basho.com/riak/latest/references/appendices/MapReduce-Implementation/
should be improved to give this warning as well. I've added an issue
to our new docs repo to track this improvement:
https://github.com/basho/basho_docs/issues/13

-Bryan




More information about the riak-users mailing list