Simpler, many m/r phases, or complex, fewer m/r phases?

Alexander Sicular siculars at gmail.com
Wed Mar 14 22:02:32 EDT 2012


I would probably say complex/fewer MR phases but I guess it would depend on the compute complexity of your functions (in order to take advantage of parallelism/more compute cores). My reasoning is that every time you Map you are reading the full value from disk. More Maps = more disk i/o. Not to mention the erlang to js overhead if you are running js functions.

Please report your findings!

Best,

-Alexander Sicular

@siculars

On Mar 14, 2012, at 6:16 PM, Jonathan Langevin wrote:

> What is better for performance in Riak?
> More phases with simpler logic, or less phases with more complex logic?
> 
> For instance, if I want to check 10 different fields of the result objects, using 10 different functions, should I combine that all down into 1-2 m/r phases, or run as 10 different m/r phases?
> 
> I would think more phases would suggest that the workload could be distributed across various nodes more easily, but few phases would mean that the values wouldn't have to be processed as many times...
> 
> 
> Jonathan Langevin
> Manager, Information Technology
> Loom Inc.
> Wilmington, NC: (910) 241-0433 - jlangevin at loomlearning.com - www.loomlearning.com - Skype: intel352
> 
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20120314/25e44136/attachment.html>


More information about the riak-users mailing list