map/reduce phases

francisco treacy francisco.treacy at gmail.com
Mon Feb 8 05:46:41 EST 2010


Kevin,

> This is a difference between Riak's map/reduce and other document stores, namely Couch. Riak map functions expect to receive bucket/key pairs as inputs to the function. Riak examines the bucket/key pairs, determines where the data lives, and then invokes the function on the node hosting the data, That's how we get map parallelism and exploit data locality in one fell swoop.

Yes, I understand all this.

> You could restructure the job as a map function, invoking Riak.mapValuesJson, followed by a reduce function using your anonymous function. Reduce functions can accept arbitrary input -- as long as the input is a list (or array depending on the language used) -- and doesn't share map functions' input restrictions.

But it's still unclear to me why should I use a reduce function in that case.

I'll explain again what I want to achieve. The following JSON works
exactly as I want:

{"inputs": "books",
  "query":[
 {"map":{"language":"javascript","source":"function(v) { return
[JSON.parse(v.values[0].data).object.chapters[4].title ]; }"}}
 ]}

It returns all the 5th chapter's titles of every book in the bucket. Perfect.

I thought, however, that invoking a phase with Riak.mapValuesJson
would help me avoid the "JSON.parse(v.values[0].data)" part... that's
it.  Is this the intended purpose of mapValuesJson or am I confused?

Thanks,

Francisco



> On Feb 7, 2010, at 5:17 PM, francisco treacy wrote:
>
>> I am able to get a sensible result out of this query:
>>
>> {"inputs": [["test", "jsondoc"]],
>>  "query":[
>> {"map":{"language":"javascript","source":"function(v){ return [v]; }"}}
>> ]}
>>
>> (just applying an identity function), however when I add a previous
>> phase to map to my objects via Riak.mapValuesJson, like:
>>
>> {"inputs": [["test", "jsondoc"]],
>>  "query":[ {"map":{"language":"javascript", "name":"Riak.mapValuesJson"}},
>> {"map":{"language":"javascript","source":"function(v){ return [v]; }"}}
>> ]}
>>
>> ...I always get back the value [].
>>
>> If I *only* apply Riak.mapValuesJson, it works as advertised. Problem
>> is with both... What could I be doing wrong?
>>
>> Thanks!
>>
>> Francisco
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>



More information about the riak-users mailing list