riak_kv_multi_backend + custom backend

Sean Cribbs sean at basho.com
Sun Jan 23 14:50:32 EST 2011


Anthony,

This is something I discovered a while back - define your backend names as binaries and you'll be able to set them properly from the REST interface. Example:

{storage_backend, riak_kv_multi_backend},
 {multi_backend_default, <<"bitcask">>},
 {multi_backend,
    [ {<<"bitcask">>, riak_kv_bitcask_backend,
        [{data_root, "/var/lib/riak/bitcask"}]},
      {<<"dets">>, riak_kv_dets_backend,
        [{riak_kv_dets_backend_root, "/var/lib/riak/dets"}]},
      {<<"ets">>, riak_kv_ets_backend, []},
      {<<"fs">>, riak_kv_fs_backend,
        [{riak_kv_fs_backend_root, "/var/lib/riak/fs"}]},
      {<<"cache">>, riak_kv_cache_backend,
        [ {riak_kv_cache_backend_memory, 100},
          {riak_kv_cache_backend_ttl, 600},
          {riak_kv_cache_backend_max_ttl, 3600}
        ]},
      {<<"my_backend">>, my_backend, []}
    ]},

Then you can set it like so

curl -X PUT -H "content-type: application/json" -d '{"props":{"backend":"my_backend"}}' http://127.0.0.1:8098/riak/mybucket

Also note that only allow_mult and n_val are supported bucket properties from the PB interface (something we'll be fixing soon).

Sean Cribbs <sean at basho.com>
Developer Advocate
Basho Technologies, Inc.
http://basho.com/

On Jan 23, 2011, at 12:37 PM, Anthony Molinaro wrote:

> 
> Hi,
> 
>  So I wanted to play around with creating a custom backend, and using
> the multi backend, but I am having problems getting anything to work.
> 
> Here's what I tried so far
> 
> in app.config
> 
>  {storage_backend, riak_kv_multi_backend},
>  {multi_backend_default, bitcask},
>  {multi_backend,
>     [ {bitcask, riak_kv_bitcask_backend,
>         [{data_root, "/var/lib/riak/bitcask"}]},
>       {dets, riak_kv_dets_backend,
>         [{riak_kv_dets_backend_root, "/var/lib/riak/dets"}]},
>       {ets, riak_kv_ets_backend, []},
>       {fs, riak_kv_fs_backend,
>         [{riak_kv_fs_backend_root, "/var/lib/riak/fs"}]},
>       {cache, riak_kv_cache_backend,
>         [ {riak_kv_cache_backend_memory, 100},
>           {riak_kv_cache_backend_ttl, 600},
>           {riak_kv_cache_backend_max_ttl, 3600}
>         ]},
>       {my_backend, my_backend, []}
>     ]},
> 
> Then I restart, open a shell and do the following
> 
> 1> {ok, Pid} = riakc_pb_socket:start_link("127.0.0.1", 8087).
> {ok,<0.68.0>}
> 2> riakc_pb_socket:set_bucket(Pid, <<"b">>, [{backend, my_backend}]).
> ok
> 3> riakc_pb_socket:get_bucket(Pid,<<"b">>).
> {ok,[{n_val,3},{allow_mult,false}]}
> 
> So I didn't see my backend there, thus tried the REST API
> 
>> curl 'http://127.0.0.1:8098/riak/b' ; echo 
> {"props":{"name":"b","n_val":3,"allow_mult":false,"last_write_wins":false,"precommit":[],"postcommit":[],"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"old_vclock":86400,"young_vclock":20,"big_vclock":50,"small_vclock":10,"r":"quorum","w":"quorum","dw":"quorum","rw":"quorum"}}
> 
> It's not there either.  So I try to set it with REST
> 
>> curl -X PUT -H "Content-Type: application/json" -d '{"props":{"backend":"my_backend"}}' http://127.0.0.1:8098/riak/b
> 
> Which works, in that now I have
> 
>> curl 'http://127.0.0.1:8098/riak/b' ; echo{"props":{"backend":"my_backend","name":"b","n_val":3,"allow_mult":false,"last_write_wins":false,"precommit":[],"postcommit":[],"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"old_vclock":86400,"young_vclock":20,"big_vclock":50,"small_vclock":10,"r":"quorum","w":"quorum","dw":"quorum","rw":"quorum"}}
> 
> However, in the shell I still get
> 
> 4> riakc_pb_socket:get_bucket(Pid,<<"b">>).
> {ok,[{n_val,3},{allow_mult,false}]}
> 
> Also, if I attempt to put something I get
> 
> 5>  riakc_pb_socket:put(Pid, riakc_obj:new (<<"b">>, <<"c">>, <<"d">>)).
> 
> =ERROR REPORT==== 19-Jan-2011::02:21:04 ===
> ** Generic server <0.68.0> terminating 
> ** Last message in was {req_timeout,#Ref<0.0.0.186>}
> ** When Server state == {state,"127.0.0.1",8087,false,false,undefined,
>                               undefined,
>                               {[],[]},
>                               1,[],infinity,100}
> ** Reason for termination == 
> ** disconnected
> ** exception exit: disconnected
> 
> =CRASH REPORT==== 19-Jan-2011::02:21:04 ===
>  crasher:
>    initial call: riakc_pb_socket:init/1
>    pid: <0.68.0>
>    registered_name: []
>    exception exit: disconnected
>      in function  gen_server:terminate/6
>    ancestors: [<0.65.0>]
>    messages: [{tcp,#Port<0.816>,
>                       [0|<<10,7,116,105,109,101,111,117,116,16,1>>]}]
>    links: [<0.65.0>]
>    dictionary: []
>    trap_exit: false
>    status: running
>    heap_size: 987
>    stack_size: 24
>    reductions: 1541
>  neighbours:
>    neighbour: [{pid,<0.65.0>},
>                  {registered_name,[]},
>                  {initial_call,{erlang,apply,2}},
>                  {current_function,{gen,do_call,4}},
>                  {ancestors,[]},
>                  {messages,[{#Ref<0.0.0.185>,{error,timeout}}]},
>                  {links,[<0.25.0>,<0.68.0>]},
>                  {dictionary,[]},
>                  {trap_exit,false},
>                  {status,runnable},
>                  {heap_size,1597},
>                  {stack_size,38},
>                  {reductions,17388}]
> 
> And the attached sasl log which has many errors.  So I'm trying to figure out
> whether the problem is configuration, or a bug or what, and wondering if any
> one else has gotten a custom backend, or multi backend to work?
> 
> I'm using 0.14 on Centos 5, using the basho RPM.
> 
> Thanks,
> 
> -Anthony
> 
> -- 
> ------------------------------------------------------------------------
> Anthony Molinaro                           <anthonym at alumni.caltech.edu>
> <sasl-error.log>_______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com




More information about the riak-users mailing list