Strange MapReduce behavior

Bryan Fink bryan at basho.com
Wed Oct 26 16:18:23 EDT 2011


On Wed, Oct 26, 2011 at 2:53 PM, Elias Levy <fearsome.lucidity at gmail.com> wrote:
> For instance:
>
> mr = Riak::MapReduce.new(riak)
> mr.add('bucket', 'key')
> mr.reduce("function(v) { return [] }", :keep => true))
> This returns:
> {"bucket"=>"key"}

I believe this one is explainable by a known Riak 1.0 MapReduce bug:

https://issues.basho.com/show_bug.cgi?id=1185

If you check your Riak log, I bet you'll see an error message, and if
the final run of a reduce fails, then in Riak 1.0.[0,1] the reduce
phase sends on its [possibly unreduced] inputs, instead of failing.  I
hope to fix this soon.

> Doing instead:
> mr = Riak::MapReduce.new(riak)
> mr.add('bucket', 'key')
> mr.reduce("riak_kv_mapreduce:reduce_identity", :language => "erlang")
> mr.reduce("function(v) { return [] }", :keep => true))
> results in the same output.

This, I'm unable to explain, unless the syntax for calling
riak_kv_mapreduce:reduce_identity is incorrect in your example (I'm
unfamiliar with Ripple's API).  If I submit what I think is the
equivalent query directly over HTTP or the Erlang console, I see the
result come back as [].  If the given syntax is incorrect, then it's
likely generating another error that is causing the same behavior as
described in Bugzilla issue 1185.

> Can't a reduce phase appear before a map phase?

A reduce phase *can* appear before a map phase, but not if the reduce
phase is implemented in Javascript and the inputs are bucket-key
pairs.  Unfortunately, bucket-key pairs are represented as tuples that
are not correctly converted to JSON, so evaluation of the phase fails.
 The typical workaround is as you attempted in your second example:
using riak_kv_mapreduce:reduce_identity to alter the representation of
bucket-key pairs (or implementing the reduce phase in Erlang instead).

-Bryan



More information about the riak-users mailing list