Weird MapReduce Behavior

Yehuda Zargrov Yehuda at conduit.com
Tue Nov 29 04:26:32 EST 2011


Hi All,

We are facing a very strange map-reduce behavior.
We use ripple in ruby, this is the call:

  def yehuda
    query_result = Riak::MapReduce.new(Ripple.client).add('usage-test1')
     .map("function(v) { data = JSON.parse(v.values[0].data).loc; for (var a in data) { for (var b in data[a]) { for (var c in data[a][b]) { return [{date:v.key, sum:data[a][b][c]}]; } } } }")
     .reduce("function(v) {var s={}; for(var i in v)  { var date=''; var sum=0; for(var n in v[i]) { if (n === \"date\") date=v[i][n]; if (n === \"sum\") sum=v[i][n]; } if (date in s) s[date] += sum; else s[date] = sum; } return[v]; }", :keep => true).run

    puts query_result
  end

The map function, though looks complicated, works fine. It gives the reduce an array of hashes like this:
{"date"=>"CT16-20110114", "sum"=>1}
{"date"=>"CT18-20101204", "sum"=>1}
{"date"=>"CT19-20110314", "sum"=>1}
{"date"=>"CT116-20100516", "sum"=>1}
{"date"=>"CT17-20110214", "sum"=>1}
{"date"=>"CT19-20100511", "sum"=>1}
{"date"=>"CT18-20100710", "sum"=>1}
{"date"=>"CT19-20110301", "sum"=>1}
{"date"=>"CT17-20110213", "sum"=>1}

There are a lot of items in this array (hundreds).
The reduce function should sum up the total sum of every date.
>From some reason it returns only 13 results:
{""=>0, "CT18-20101222"=>1, "CT116-20101123"=>1, "CT18-20101208"=>1, "CT18-20101110"=>1, "CT116-20101028"=>1, "CT18-20100904"=>1, "CT18-20100820"=>1, "CT18-20100618"=>1, "CT17-20110420"=>1, "CT17-20110412"=>1, "CT17-20110407"=>1, "CT17-20110401"=>1}.

Does someone have a clue why?

Thanks,

Yehuda Zargarov



[cid:image001.gif at 01CCAE89.BEE19870]

Yehuda Zargrov
Software Engineer
Conduit Your Site <http://www.conduit.com/>
t: 972.8.9461713 ext +314 |   m: +972503473119   |   f: +97286382140


This message may contain confidential and/or privileged information. It is intended to be read only by the individual or entity to whom it is addressed or by their designee.
If the reader of this message is not the intended recipient, you are on notice that any uses, copy, disclose or distribution of this message, in any form, is strictly prohibited.
If you have received this message in error, please notify the sender by reply email or by telephone and delete or destroy any copy of this message. Thank you.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20111129/6fb9995e/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 1930 bytes
Desc: image001.gif
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20111129/6fb9995e/attachment.gif>


More information about the riak-users mailing list