"consistent" map/reduce

Kresten Krab Thorup krab at trifork.com
Wed Nov 30 16:56:40 EST 2011


Thanks for your answers.

Now, it seems that if the items passing between the map phase and reduce phase are many and small, then the system could benefit from a "chunking" fitting that collects items and sends them off in a list to the next fitting, after receiving X # of items, or after some timeout (say, Y ms); i.e. basically Nagle's algorithm  Otherwise, we may get burned pretty seriously by the "sync send" that happens between fittings !?

So ...

{
   list_keys,  
   riak_kv_get, follow
   riak_kv_mrc_map, follow
   nagles_send, follow   

   nagles_receive, riak_pipe_w_reduce:chashfun
   riak_pipe_w_reduce, follow
}



   

On Nov 30, 2011, at 4:08 PM, Bryan Fink wrote:

> Ew.  Major apologies for the formatting in my last response.  I've no
> idea what happened.  Let me know if it's unreadable, and I'll try
> sending again.
> 
> -Bryan





More information about the riak-users mailing list