Riak mapreduce function map_object_value return empty result on erlang pb client

Mikhail Kuznetsov kuznetsov.m.yu at gmail.com
Mon Oct 15 12:34:14 EDT 2012


Hi 
Thank you Brian. Today we finally find problems 
We use index fetch before use mapreduce, like in this example:

{ok,List}=riakc_pb_socket:get_index(Pid, Bucket, Index, Key)
BK=[ {B,K}||[B,K]<-List],
{ok,[{0,Result}]}=riakc_pb_socket:mapred(Pid, BK, [{map, {modfun, riak_kv_mapreduce, map_object_value} ,undefined, true}]),
[binary_to_term(E)|| E<-Result]

In line 1 function return list of lists (pair Bucket and Key). And after some time that function start returning simply list of keys. Without announcement, release notes or ticket or something else. Just one day one man think why not to rewrite everything related to search. Right before it someone "fixed" protobuff and we got tons of errors related to binary indexes. What the fuck, we spend one day to check driver source for possible bugs? I think main commiter at least should post some notes and not to change function interfaces without some serious need. 

Erlang VM in some case use it pattern matching without runtime error in line 3 and we got our silent bug. All we need was to change line 3 to
BK=[ {Bucket,K} || K<-List],

It works fine now. But this is not ok. Because interface is broke for map reduce function. We pass it empty list instead of list of {Bucket, Key} and we get {ok,[]} instead of {ok, [{0,[]}]}.
We fixed it, but I am very angry to people who don't give a fuck about interface stability and release notes. Many people use it in production, they pay money for support and a lot of money. They don't want to play in hide and seek "new features" after each minor commit. If they feel pissed they just pull out  they money and buy something else.
-- 
Sincerely yours,
Mikhail Kuznetsov

When best practices meet everyday life and lead to perfection...

Oct 15, 2012, в 19:55 , Bryan Fink написал(а):

> On Thu, Oct 11, 2012 at 1:32 AM, Mikhail Kuznetsov
> <kuznetsov.m.yu at gmail.com> wrote:
>> I use riak erlang client for my project, and update riak and client library
>> today from master(client to 1.3.1 and server to 1.2.0). Everything works
>> except fetching multiple entities with map-reduce return empty. Instead of
>> usual result {ok, [...,{<stage_number>,<stage result>},...]} I get {ok,[]}.
>> Rollback with client and everything works ok. But if I use old client with
>> new riak I got a lot of errors in the server logs.
> 
> Hi, Mikhail. I'm not surprised by the old-client-with-new-riak errors.
> The definitions of some protocol buffers changed in the 1.2.0 Riak
> release, iirc.
> 
> As for the missing results, though, I've been unable to reproduce the
> issue as you reported. Using Riak 1.2.0 and riak-erlang-client 1.3.1:
> 
> 
> 1> {ok, S} = riakc_pb_socket:start("127.0.0.1", 8087).
> {ok,<0.33.0>}
> 
> 2> BK = [{<<"mk">>,<<"1">>},{<<"mk">>,<<"2">>},{<<"mk">>,<<"3">>}].
> [{<<"mk">>,<<"1">>},{<<"mk">>,<<"2">>},{<<"mk">>,<<"3">>}]
> 
> 3> [ riakc_pb_socket:put(S, riakc_obj:new(B, K, K)) || {B, K} <- BK ].
> [ok,ok,ok]
> 
> 4> {ok, [{0,Result}]} = riakc_pb_socket:mapred(S, BK, [{map, {modfun,
> riak_kv_mapreduce, map_object_value},undefined, true}]).
> {ok,[{0,[<<"1">>,<<"2">>,<<"3">>]}]}
> 
> Can you see where my example differs from yours?
> 
> -Bryan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20121015/4cdf4530/attachment.html>


More information about the riak-users mailing list