map/reduce phases

Kevin Smith ksmith at basho.com
Sun Feb 7 19:59:20 EST 2010


Francisco -

This is a difference between Riak's map/reduce and other document stores, namely Couch. Riak map functions expect to receive bucket/key pairs as inputs to the function. Riak examines the bucket/key pairs, determines where the data lives, and then invokes the function on the node hosting the data, That's how we get map parallelism and exploit data locality in one fell swoop.

You could restructure the job as a map function, invoking Riak.mapValuesJson, followed by a reduce function using your anonymous function. Reduce functions can accept arbitrary input -- as long as the input is a list (or array depending on the language used) -- and doesn't share map functions' input restrictions.

--Kevin
On Feb 7, 2010, at 5:17 PM, francisco treacy wrote:

> I am able to get a sensible result out of this query:
> 
> {"inputs": [["test", "jsondoc"]],
>  "query":[
> {"map":{"language":"javascript","source":"function(v){ return [v]; }"}}
> ]}
> 
> (just applying an identity function), however when I add a previous
> phase to map to my objects via Riak.mapValuesJson, like:
> 
> {"inputs": [["test", "jsondoc"]],
>  "query":[ {"map":{"language":"javascript", "name":"Riak.mapValuesJson"}},
> {"map":{"language":"javascript","source":"function(v){ return [v]; }"}}
> ]}
> 
> ...I always get back the value [].
> 
> If I *only* apply Riak.mapValuesJson, it works as advertised. Problem
> is with both... What could I be doing wrong?
> 
> Thanks!
> 
> Francisco
> 
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com





More information about the riak-users mailing list