erlang precommit hook error

Hal Eisen eisen at dunhackin.org
Wed Nov 9 23:28:39 EST 2011


My input data is ascii text (not json or xml).  Why would
riak_object:get_value() return binary data?

Can anyone explain how to retrieve, modify and store a riak_object
within an Erlang precommit hook?

I'm also trying to get some debug info with lager:info() but that seems
to be giving an error as well.

Thanks,
Hal

On Thu, 2011-11-10 at 10:17 +0800, Joseph Lambert wrote:
> AFAIK,
> 
> 
> When you do riak_object:get_value(), it will return a binary and you
> are calling string:tokens on a binary value.
> 
> - Joe Lambert
> 
> joseph.g.lambert at gmail.com
> +86 13656213284
> 
> 
> On Wed, Nov 9, 2011 at 4:01 AM, Hal Eisen <eisen at dunhackin.org> wrote:
>         I've made some progress.  Looks like I was not compiling my
>         module,
>         nor did I have the add_paths set in app.config.
>         
>         However, now I'm getting a new failure:
>          error:function_clause
>         
>         When I look in my erlang.log file, I see this:
>         [{string,'tokens1',[<<"my actual data that I am trying to
>         store before being processed by the precommit hook">>,"
>         ",[]]},{haleisen,mangle,1},{riak_kv_put_fsm,invoke_hook,4},{riak_kv_put_fsm,precommit,2},{gen_fsm,handle_msg,7},{proc_lib,init_p_do_apply,3}]
>         
>         
>         I've updated my mangle/1 function:
>         mangle(X) ->
>            crypto:start(),
>            {_, Tokens} = lists:partition(fun(Arg) -> lists:member(Arg,
>         haleisen:stopwords()) end,
>         string:tokens(riak_object:get_value(X), " ")),
>            riak_object:update_value(X,
>         list_to_binary(lists:flatten(lists:map(fun(Arg) ->
>         crypto:md5(Arg) end, Tokens)))),
>            riak_object:apply_updates(X).
>         
>         Am I manipulating the riak_object correctly?
>         
>         Is there a recommended way to emit debug message from within
>         Erlang hooks?
>         
>         Thanks,
>         Hal
>         
>         On Tue, Nov 08, 2011 at 10:01:26AM -0800, Hal Eisen wrote:
>         > Sorry, typo in code.  the mangle function is actually:
>         >
>         > mangle(X) ->
>         >     crypto:start(),
>         >     {_, Tokens} = lists:partition(fun(Arg) ->
>         lists:member(Arg, mangle:stopwords()) end,
>         string:tokens(riak_object:get_value(X), " ")),
>         >     riak_object:update_value(X,
>         lists:flatten(lists:map(fun(Arg) -> crypto:md5(Arg) end,
>         Tokens))),
>         >     riak_object:apply_updates(X).
>         >
>         > In other words, I fixed the module reference.
>         >
>         > Hal
>         >
>         > On Tue, Nov 08, 2011 at 09:59:26AM -0800, Hal Eisen wrote:
>         > > Hello again.  I have given up on using the JavaScript
>         precommit hook
>         > > because I could not find anything which did not produce
>         timeouts under
>         > > the load for my application.
>         > >
>         > > So, I have ported my hook to Erlang.  Alas, I'm not
>         getting very far.
>         > > I've installed my hook in /etc/riak/erl/hooks.erl.  The
>         code looks
>         > > like this:
>         > >
>         > > -module(haleisen).
>         > > -export([stopwords/0, mangle/1]).
>         > > stopwords() ->
>         > >
>         ["a","able","about","across","after","all","almost","also","am","among","an","and","any","are","as","at","be","because","been","but","by","can","cannot","could","dear","did","do","does","either","else","ever","every","for","from","get","got","had","has","have","he","her","hers","him","his","how","however","i","if","in","into","is","it","its","just","least","let","like","likely","may","me","might","most","must","my","neither","no","nor","not","of","off","often","on","only","or","other","our","own","rather","said","say","says","she","should","since","so","some","than","that","the","their","them","then","there","these","they","this","tis","to","too","twas","us","wants","was","we","were","what","when","where","which","while","who","whom","why","will","with","would","yet","you","your"].
>         > > mangle(X) ->
>         > >     crypto:start(),
>         > >     {_, Tokens} = lists:partition(fun(Arg) ->
>         lists:member(Arg, ask:stopwords()) end,
>         string:tokens(riak_object:get_value(X), " ")),
>         > >     riak_object:update_value(X,
>         lists:flatten(lists:map(fun(Arg) -> crypto:md5(Arg) end,
>         Tokens))),
>         > >     riak_object:apply_updates(X).
>         > >
>         > > I originally tested the code in the Erlang repl, using a
>         plain string
>         > > instead of riak_object:get_value, and it worked perfectly.
>         > >
>         > > I installed my hook into a test bucket:
>         > > bash$ curl http://127.0.0.1:8098/riak/test-hook
>         > >
>         {"props":{"allow_mult":false,"basic_quorum":false,"big_vclock":50,"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"dw":"quorum","last_write_wins":false,"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"n_val":3,"name":"test-with","notfound_ok":true,"old_vclock":86400,"postcommit":[],"pr":0,"precommit":[{"mod":"haleisen","fun":"mangle"}],"pw":0,"r":"quorum","rw":"quorum","small_vclock":10,"w":"quorum","young_vclock":20}}
>         > >
>         > > When I try to insert a key into this bucket, I get:
>         > > Error:
>         > >
>         {precommit_fail,{hook_crashed,{haleisen,mangle,error,undef}}}
>         > >
>         > > Any help would be appreciated.
>         > >
>         > > Thanks,
>         > > Hal
>         
>         _______________________________________________
>         riak-users mailing list
>         riak-users at lists.basho.com
>         http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>         
> 
> 






More information about the riak-users mailing list