Map Reduce and long queries -

Olav Frengstad olav at fwt.no
Mon Oct 15 11:35:36 EDT 2012


I use ``make_local_fun`` to send the fun's across the cluster:
https://gist.github.com/510070

In my opinion you should be able to send Erlang string fun's, and use
erl_eval to parse them. Unfortunately this is also allows for
execution of any erlang code so security wise people might not want
it, maybe there is a way to create a sandbox or only expose certain
modules?

Olav

2012/10/15, Bryan Fink <bryan at basho.com>:
> On Mon, Oct 15, 2012 at 4:13 AM, Olav Frengstad <olav at fwt.no> wrote:
>> Just as a note, using the Erlang pb client you can use the key filters
>> for 2i queries if you include the riak_kv_mapred_filters module in
>> your client code path.
>>> 2> Index = {index, <<"test">>, <<"$key">>, <<0>>, <<255>>},
>> 2> {ok, Filter} =
>> riak_kv_mapred_filters:build_filter([[<<"ends_with">>,"1"]]),
>> 2> MR = [
>> 2>   { reduce
>> 2>   , {qfun, fun(X, F) -> lists:filter(fun({A, B}) -> F(B) end, X) end}
>> 2>   , riak_kv_mapred_filters:compose(Filter)
>> 2>   , true}],
>> 2> riakc_pb_socket:mapred(Pid, Index, MR).
>
> It's a bit unfortunate that qfun is so useful, because it's also so
> fragile. If your client node is not using exactly the same module
> version as every node in your Riak cluster, this will fail with
> undefined function errors. Absolutely, go ahead and use qfun to learn
> with, but keep an eye out for that kind of surprise. Avoid qfun in
> production if you can, or your upgrade process will become more
> complex and/or prone to failure.
>
> Yes,
> http://docs.basho.com/riak/latest/references/appendices/MapReduce-Implementation/
> should be improved to give this warning as well. I've added an issue
> to our new docs repo to track this improvement:
> https://github.com/basho/basho_docs/issues/13
>
> -Bryan
>


-- 
Med Vennlig Hilsen
Olav Frengstad

Systemutvikler // FWT
+47 920 42 090




More information about the riak-users mailing list