Problems with weird errors running 0.13

Dan Reverri dan at basho.com
Mon Nov 29 13:15:15 EST 2010


Hi Jon,

It looks like Riak 0.13 may be running an initial empty reduce phase; we'll
look into this issue. With that said, the reduce function you have defined
does not account for "re-reduce". The following article explains how reduce
phases work:
https://wiki.basho.com/display/RIAK/MapReduce#MapReduce-HowaReducePhaseWorksinRiak

Note the following from the article:
"The important thing to understand is that the function defining the reduce
phase may be evaluated multiple times, and the input of later evaluations
will include the input of earlier evaluations."

Based on your defined reduce function it looks like you are trying to count
the number of objects in the "test" bucket. Rather than return the length of
the incoming list it would be better to sum the values of the incoming list
using "reduceSum":
https://github.com/basho/riak_kv/blob/master/priv/mapred_builtins.js#L66

Thanks,
Dan

Daniel Reverri
Developer Advocate
Basho Technologies, Inc.
dan at basho.com


On Mon, Nov 29, 2010 at 7:08 AM, Jon Brisbin <
jon.brisbin at npcinternational.com> wrote:

>
> On Nov 29, 2010, at 8:59 AM, David Smith wrote:
>
> > On Mon, Nov 29, 2010 at 7:43 AM, Jon Brisbin
> > <jon.brisbin at npcinternational.com> wrote:
> >> I'm working on the Spring Data and Grails Gorm support for Riak and I'm
> seeing some problems running my tests against Riak 0.13. I don't see these
> problems when running 0.12.
> >
> > Can you elaborate on the problems you're seeing, please?
>
> There's two problems I'm seeing, one is with map/reduce, the other is that
> the server will crash under the load the tests put on it (I'll send a
> separate email with steps to reproduce).
>
> Here's an example of a problem I'm seeing in M/R (from an earlier message):
>
> I'm seeing some weird behavior with map/reduce in 0.13 versus 0.12 and I'm
> not sure what's going on here. I'm hanging out in the chat room right now
> trying to find out, but basically, if I run this test case against 0.13 I
> get a completely different result than what I get in 0.12:
>
> http://pastie.org/1320874
>
> In 0.12, I get [1] in 0.13 I get [2].
>
> The output I'm logging looks like this for 0.12:
>
> 11/23/2010 (13:39:47):
> map={"bucket":"test","key":"test","vclock":"a85hYGBgzGDKBVIsTE+lb2UwJTLmsTLUTEk4xpcFAA==","values":[{"metadata":{"Links":[],"X-Riak-VTag":"5qaBBTXfdEQUyCBTDRRIQ7","content-type":"application/json","X-Riak-Last-Modified":"Tue,
> 23 Nov 2010 19:39:40
> GMT","X-Riak-Meta":[]},"data":"{\"test\":\"value\",\"integer\":12}\n"}]}
> 11/23/2010 (13:39:47): reduce=[1]
>
> And this in 0.13:
>
> 11/23/2010 (13:46:52):
> map={"bucket":"test","key":"test","vclock":"a85hYGDgzmDKBVIsjDI+7RlMiYx5rAz9ExKO8UGE2ZqTWGZL10AlRCfBJVgYLplNxhQGqmeMbn4EldBEUs/k7LodUxionils9l2oRBGy+cHJ7zGFgeqZk3faQCVOT0aoZzG1PohFmNWr8CxUWGUaijFv+lYiS2QBAA==","values":[{"metadata":{"Links":[],"X-Riak-VTag":"If4qIgutNTerbEmxiKLhR","content-type":"application/json","X-Riak-Last-Modified":"Tue,
> 23 Nov 2010 19:46:44
> GMT","X-Riak-Meta":[]},"data":"{\"test\":\"value\",\"integer\":12}\n"}]}
> 11/23/2010 (13:46:52): reduce=[]
> 11/23/2010 (13:46:52): reduce=[1,0]
>
> I tried Homebrew (OS X 10.6) and the OS X download packages of both
> versions.
>
> Jon Brisbin
> Portal Webmaster
> NPC International, Inc.
>
>
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20101129/346ee55f/attachment.html>


More information about the riak-users mailing list