running erlang map phases via REST API

Brendan brendan at tucows.com
Mon Jan 24 15:56:56 EST 2011


having an anonymous fun would seem useful on a larger cluster - you
wouldn't need to distribute a new file/module to all machines in the
cluster (as i think you must do now?).

the docs around calling an erlang map phase are a bit confusing
(http://wiki.basho.com/MapReduce.html):
> Function source can be specified directly in the query by using the
> "source" spec field. Function source can also be loaded from a
> pre-stored riak object by providing "bucket" and "key" fields in the
> spec. Erlang map functions can be specified using the "module" and
> "function" fields in the spec.
it wasn't immediately obvious to me that the erlang functions *must* be
specified with module+function, especially since an anonymous fun is
used later on as an example in the same document.

the wiki doc also doesn't indicate where to put any custom erlang
map/reduce functions... do they go into riak-0.14.0/deps/riak_kv/src/
and then need to be built into the release?



On 11-01-24 12:34 PM, Dan Reverri wrote:
> Hi Brendan,
>
> Anonymous Erlang functions are not currently supported in map reduce
> phases. Is this a feature the community would find valuable?
>
> Thanks,
> Dan
>
> Daniel Reverri
> Developer Advocate
> Basho Technologies, Inc.
> dan at basho.com <mailto:dan at basho.com>
>
>
> On Mon, Jan 24, 2011 at 12:14 PM, Brendan <brendan at tucows.com
> <mailto:brendan at tucows.com>> wrote:
>
>     is it possible to pass erlang anonymous funs via the REST API? i
>     took a
>     javascript query, replaced "language":"javascript" with
>     "language":"erlang" and changed the source to an anonymous fun, but i
>     end up getting an error from the REST API back.am <http://back.am>
>     i doing something
>     wrong here, or can erlang functions only be called using the "module"
>     and "function" fields?
>
>     the anonymous erlang fun was pulled from the Phase Functions->Map
>     Function examples section of the http://wiki.basho.com/MapReduce.html
>     wiki page.
>
>     [dev.a]brendan at build01:~/riak$ curl -X PUT -d 'stuff'
>     http://127.0.0.1:8098/riak/bucket/object
>     [dev.a]brendan at build01:~/riak$ curl -X GET
>     http://127.0.0.1:8098/riak/bucket/object; echo
>     stuff
>     [dev.a]brendan at build01:~/riak$ cat erl2
>     {
>        "inputs": [
>            [
>                "bucket",
>                "object"
>            ]
>        ],
>        "query": [
>            {
>                "map": {
>                    "language": "erlang",
>                    "source":
>     "fun(Value,_Keydata,_Arg)->[[riak_object:get_value(Value)]] end."
>                }
>            }
>        ]
>     }
>     [dev.a]brendan at build01:~/riak$ curl -X POST -H
>     "content-type:application/json" http://localhost:8098/mapred
>     --data @erl2
>     <html><head><title>500 Internal Server
>     Error</title></head><body><h1>Internal Server Error</h1>The server
>     encountered an error while processing this
>     request:<br><pre>{error,badarg,
>           [{erlang,binary_to_list,[undefined]},
>            {riak_kv_mapred_json,bin_to_atom,1},
>            {riak_kv_mapred_json,parse_step,2},
>            {riak_kv_mapred_json,parse_query,2},
>            {riak_kv_mapred_json,parse_request,1},
>            {riak_kv_wm_mapred,verify_body,2},
>            {riak_kv_wm_mapred,malformed_request,2},
>
>     {webmachine_resource,resource_call,3}]}</pre><P><HR><ADDRESS>mochiweb+webmachine
>     web server</ADDRESS></body></html>
>
>
>
>     _______________________________________________
>     riak-users mailing list
>     riak-users at lists.basho.com <mailto:riak-users at lists.basho.com>
>     http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110124/bd32068e/attachment.html>


More information about the riak-users mailing list