Simpler, many m/r phases, or complex, fewer m/r phases?
jlangevin at loomlearning.com
Wed Mar 14 22:25:58 EDT 2012
k, that's what I was afraid of (full value read from disk).
This is for a client lib for Riak, so I'll reconfigure it to combine the
logic down to single map/reduce phases. Luckily the logic in this case is
mostly non-complex, while custom m/r functionality would be sent per phase
specified by the application. The custom m/r phases would likely be more
complex logic, so that pretty much works itself out.
Thanks for the feedback!
Manager, Information Technology
Wilmington, NC: (910) 241-0433 - jlangevin at loomlearning.com -
www.loomlearning.com - Skype: intel352
On Wed, Mar 14, 2012 at 10:02 PM, Alexander Sicular <siculars at gmail.com>wrote:
> I would probably say complex/fewer MR phases but I guess it would depend
> on the compute complexity of your functions (in order to take advantage of
> parallelism/more compute cores). My reasoning is that every time you Map
> you are reading the full value from disk. More Maps = more disk i/o. Not to
> mention the erlang to js overhead if you are running js functions.
> Please report your findings!
> -Alexander Sicular
> On Mar 14, 2012, at 6:16 PM, Jonathan Langevin wrote:
> What is better for performance in Riak?
> More phases with simpler logic, or less phases with more complex logic?
> For instance, if I want to check 10 different fields of the result
> objects, using 10 different functions, should I combine that all down into
> 1-2 m/r phases, or run as 10 different m/r phases?
> I would think more phases would suggest that the workload could be
> distributed across various nodes more easily, but few phases would mean
> that the values wouldn't have to be processed as many times...
> Jonathan Langevin
> Manager, Information Technology
> Loom Inc.
> Wilmington, NC: (910) 241-0433 - jlangevin at loomlearning.com -
> www.loomlearning.com - Skype: intel352
> riak-users mailing list
> riak-users at lists.basho.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the riak-users