riak2 erlang mapreduce counters

Bryce Verdier bryceverdier at gmail.com
Fri Jan 24 13:53:05 EST 2014


Hi Russell,

Thank you for the code, it worked like a charm.

Warm regards,
Bryce


On 01/23/2014 11:59 PM, Russell Brown wrote:
> Hi Bryce,
>
> Sorry about this, and thanks for the detailed info. I do need to add MapReduce friendly functions.
>
> On 23 Jan 2014, at 23:51, Bryce Verdier <bryceverdier at gmail.com> wrote:
>
>> Thank you both Eric & Russeli for the answer, sadly it leads to more questions. Regardless of the type (though I can say in this case the counters were pushed from the python 2.0.2 client, so I assume its riak_dt_pncounter)
> It is a riak_kv_pncounter, I don’t think any of the clients support the new API end points (bucket types) so you’ll be using the 1.4 counter.
>
>> I get this error:
>> {"phase":0,"error":"badarg","input":"{ok,{r_object,<<\"ogir-fp\">>,<<\"682l2fp6\">>,[{r_content,{dict,4,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[],[],[],[],[],[],[[<<\"content-type\">>,97,112,112,108,105,99,97,116,105,111,110,47,114,105,97,107,95,99,111,117,110,116,101,114],[<<\"X-Riak-VTag\">>,52,103,66,88,71,122,56,55,111,105,66,112,103,65,75,99,54,72,55,69,110,79]],[[<<\"index\">>]],[],[[<<\"X-Riak-Last-Modified\">>|{1390,508268,984013}]],[],[]}}},<<69,1,71,1,0,0,0,29,70,1,131,108,0,0,0,1,104,2,109,...>>}],...},...}","type":"error","stack":"[{erlang,apply,[<<\"riak_kv_pncounter\">>,new,[]],[]},{riak_kv_crdt,crdt_value,2,[{file,\"src/riak_kv_crdt.erl\"},{line,94}]},{riak_kv_crdt,value,2,[{file,\"src/riak_kv_crdt.erl\"},{line,86}]},{mr_kv_counters,value,3,[{file,\"mr_kv_counters.erl\"},{line,38}]},{riak_kv_mrc_map,map,3,[{file,\"src/riak_kv_mrc_map.erl\"},{line,165}]},{riak_kv_mrc_map,process,3,[{file,\"src/riak_kv_mrc_map.erl\"},{line,141}]},{riak_pipe_vnode_worker,process_input,3,[{file,\"src/riak_pipe_vnode_worker.erl\"},{line,445}]},{riak_pipe_vnode_worker,...}]”}
> That binary (<<69,1,71…>>) says CRDT(69), Version 1(1), riak_kv_pncounter (71). The error says that erlang is trying to apply the Module, Function, Arguments of <<“riak_kv_pncounter”>>, new, []. But modules need to be atoms.
>
>> Just to help, this is my erlang MR code:
>> value(RiakObject, _KeyData, _Arg) ->
>>     Key   = riak_object:key(RiakObject),
>>     Count = riak_kv_crdt:value(RiakObject, <<"riak_kv_pncounter">>),
> Type needs to be an atom, as it is the module name (so riak_kv_pncounter). Also, the return value is now a tuple of
>
>      {{Context :: binary(), Value :: term()}, Stats :: proplist()}. You probably only want the Value bit. So:
>
>      {{_Ctx, Count}, _Stats} = riak_kv_crdt:value(RiakObject, riak_kv_pncounter),
>
> Should be what you need, let me know if that works, please?
>
> Cheers
>
> Russell
>
>>     [ {Key, Count} ].
>>
>> What am I doing wrong? I can't seem to figure it out... I'm sure its something simple thing I'm just not seeing.
>>
>> Thanks again,
>> Bryce
>>
>> On 01/23/2014 01:07 PM, Russell Brown wrote:
>>> On 23 Jan 2014, at 20:51, Eric Redmond <eredmond at basho.com> wrote:
>>>
>>>> For version 1.4 counters, riak_kv_pncounter. For 2.0 CRDT counters, riak_dt_pncounter.
>>> As in, if the data was written in 1.4, or in 2.0 using the legacy, backwards compatible 1.4 API endpoints, the the type is risk_kv_pncounter. If the counter is 2.0, bucket types counter, then risk_dt_pncounter.
>>>
>>> Really, we need to re-introduce the riak_kv_counter module for backwards compatibility, and add some friendly `value’ functions to risk_kv_crdt. I’m opening an issue for just this now.
>>>
>>> The other option is to include the riak_kv_types.hrl and use the macros ?MAP_TYPE, ?SET_TYPE, ?V1_COUNTER_TYPE, ?COUNTER_TYPE for now, and assume that we’ll have some helper functions for MapReduce in before 2.0.
>>>
>>> Cheers
>>>
>>> Russell
>>>
>>>> Eric
>>>>
>>>> On Jan 23, 2014, at 3:44 PM, Bryce Verdier <bryceverdier at gmail.com> wrote:
>>>>
>>>>> In 1.4 there was just the simple function riak_kv_counters:value. In 2.0 I found the riak_kv_crdt module, which has a value function in it. But I'm not sure what "type" to use for second value argument for a counter.
>>>>>
>>>>> Can someone share that with me?
>>>>>
>>>>> Thanks in advance,
>>>>> Bryce
>>>>>
>>>>> _______________________________________________
>>>>> riak-users mailing list
>>>>> riak-users at lists.basho.com
>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>> _______________________________________________
>>>> riak-users mailing list
>>>> riak-users at lists.basho.com
>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com






More information about the riak-users mailing list