mochijson2 error from curl mapreduce

Bob Ippolito bob at redivi.com
Tue Jun 5 17:27:21 EDT 2012


On Tue, Jun 5, 2012 at 1:22 PM, John Roy <john at thoughtleadr.com> wrote:

> I saw something similar from an individual who was using javascript (
> http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-July/004843.html), but
> I don't think I have the same problem in erlang.  Any help is greatly
> appreciated.
>
> I'm consistently getting the following error from my curl map reduce calls:
>
> {error,{exit,{json_encode,{bad_term,…..
>
> output dictionary here, then:
>
>         [{mochijson2,json_encode,2},
>          {mochijson2,'-json_encode_array/2-fun-0-',3},
>          {lists,foldl,3},
>          {mochijson2,json_encode_array,2},
>          {riak_kv_wm_mapred,pipe_mapred_nonchunked,5},
>          {webmachine_resource,resource_call,3},
>          {webmachine_resource,do,3},
>
> The term that follows the error appears to the correct result of the map
> reduce.
>
> I'm just trying the mr locally for testing purposes.  I'm using eleveldb
> backend and riak (1.1.2 2012-04-17) OSX x86_64.
>
> Here's the curl command:
>
> curl -v -X POST http://127.0.0.1:8098/mapred -H "Content_Type:
> application/json" -d @query.js
>
> and the query.js:
>
> {
>     "inputs" : {
>         "bucket" : "test7",
>         "index" : "field1_int",
>         "start" : 0,
>         "end" : 23
>     },
>     "query" : [
>         { "map" : {"language" : "erlang", "module" : "impressions",
> "function" : "emitStats", "keep" : false} },
>         { "reduce" : {"language" : "erlang", "module" : "impressions",
> "function" : "reduceStats", "keep" : true} }
>     ],
>     "timeout" : 600000
> }
>
> and finally the associated erlang functions:
>
> emitStats(G, undefined, none)->
>     ObjectJson = riak_object:get_value(G),
>     {struct, DocPropList} = mochijson:decode(ObjectJson),
>     I = {proplists:get_value("obj_name",DocPropList,"None"),
>          proplists:get_value("type",DocPropList,"None"),
>          proplists:get_value("obj_url",DocPropList,"None"),
>          "hourly"},
>     [dict:from_list([{I,1}])].
>
> reduceStats(Gcounts, none)->
>     [lists:foldr(fun(G, Acc)->dict:merge(fun(_, X, Y) -> X+Y end,G, Acc)
> end,
>                  dict:new(),
>                  Gcounts)].
>

I don't have any experience with Riak's MR implementation, but mochijson2
does not have any native support for serializing dict, and JSON itself
doesn't support keys that are not strings, and I suspect that one or both
of those may be your problem. mochijson2 expects to see {struct, [{Key ::
binary(), Value :: json_term()}]} for JSON objects.

-bob
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20120605/01fc6587/attachment.html>


More information about the riak-users mailing list