Mapreduce crosstalk

Kelly McLaughlin kelly at basho.com
Tue May 24 12:55:30 EDT 2011


Kyle,

Just wanted to let you know that this bug is fixed in master now. If you are
interested, I describe some details about the issue and the fix in the
commit message here:
https://github.com/basho/riak_kv/commit/f7e09d54c932f24d1d06fb595ce74e686657810f

Kelly

On Tue, May 17, 2011 at 1:35 PM, Aphyr <aphyr at aphyr.com> wrote:

> I was writing a new mapreduce query to look at users over time, and ran it
> over a single user in production. After that, other mapreduce jobs over
> users started returning results from my new map phase, some of the time.
> After five minutes of this, I had to restart every node in the cluster to
> get it to stop.
>
> Every node has {map_cache_size, 0} in riak_kv.
>
> The map phase that screwed things up was:
>
> function(v) {
>  o = JSON.parse(v.values[0].data);
>
>  // Age of account in days
>  age = Math.round(
>    (Date.now() - Date.iso8601(o.created_at)) /
>    (1000 * 60 * 60 * 24)
>  );
>
>  return [['t_user_scores', v.key, age]];
> }
>
> It looks like one node started running that phase instead of the requested
> phase for subsequent jobs. It *should* have run this one, but didn't.
>
> function(v) {
>        o = JSON.parse(v.values[0].data);
>        return [{
>                key: v.key,
>                name: o.name,
>                thumbnail: o.thumbnail
>        }];
> }
>
> Now I'm scared to run MR jobs. Could it be an issue with returning keydata?
> Anybody else seen this before?
>
> --Kyle
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>



-- 
Kelly McLaughlin
Engineer
Basho Technologies, Inc.
kelly at basho.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110524/ee58e052/attachment.html>


More information about the riak-users mailing list