map reduce on multiple buckets

Alex Moore amoore at basho.com
Mon Feb 8 11:23:40 EST 2016


One possibility would be to get the intermediate results from each bucket,
and then compute the final results on the client. How much data would be
involved in the initial MR, and at the point where you would have to
combine the results?

--Alex

On Thu, Jan 28, 2016 at 12:10 PM, Eugene Shubin <ev.shuba at gmail.com> wrote:

> I want to keep daily and weekly data in different buckets, and generate
> monthly report using map reduce on these two.
> so 2 buckets and MR operations are custom.
>
> Best,
> Eugene
>
> 2016-01-28 17:29 GMT+01:00 Alex Moore <amoore at basho.com>:
>
>> Hi Eugene,
>>
>> MR is limited to one bucket for inputs, and the Solr inputs to a map
>> phase have this restriction too.
>>
>> How many buckets are you trying to MR across, and also what type of MR
>> operation are you trying to do? There may be another way to get around this
>> restriction.
>>
>> Thanks,
>> Alex
>>
>> On Thu, Jan 28, 2016 at 8:10 AM, Eugene Shubin <ev.shuba at gmail.com>
>> wrote:
>>
>>> Is it possible to run mapreduce job on two or more buckets?
>>> I see from documentation that it might be possible if I specify inputs
>>> as list of {bucket, key} pairs,
>>> although list of secondary index inputs causes an error:
>>> riakc_pb_socket:mapred(P, [
>>>     {index, Bucket1, Index1, From, To},
>>>     {index, Bucket2, Index2, From, To}
>>>   ], ...
>>>  {error,<<"{inputs,{\"Inputs target tuples must be {B,K} or
>>> {{B,K},KeyData}:\",\n
>>>
>>> Is it possible using Solr (riak search) indexes?
>>>
>>> Evgenii Shubin
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> riak-users at lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20160208/35b215f7/attachment-0002.html>


More information about the riak-users mailing list