riak_multi_backend and bitcask

Tamas Nagy tamas.nagy at erlang-solutions.com
Wed Jun 23 13:45:44 EDT 2010


Hi,

I'm pretty new to the list (and riak) so please forgive my ignorance but I didn't manage to find a public bugtracker for riak. Hence I'm posting my problem here. (Bitbucket and Internet Explorer(using it not by choice) do not mix well. So that might be the problem why I didn't find it).

I've tried to use riak_kv_multi_backend with one of the backends being the riak_kv_bitcask_backend. This does not seem to work with riak-0.11.0. I checked tip as well, and I do not think it would work either. The problem boils to these few lines (code snippets from tip):

riak_kv_vnode:

handle_info({Mod, Msg}, StateName, #state { mod = Mod } = StateData) ->
    Mod:handle_info(StateData#state.modstate, Msg),
    {next_state, StateName, StateData, ?TIMEOUT}.

riak_kv_bitcask_backend:

start(Partition, _Config) ->
    %% Schedule sync (if necessary)
    case application:get_env(bitcask, sync_strategy) of
        {ok, {seconds, Seconds}} ->
            SyncIntervalMs = timer:seconds(Seconds),
            erlang:send_after(SyncIntervalMs, self(),
                              {?MODULE, {sync, SyncIntervalMs}});
        _ ->            ok    end,
    %% Schedule merge checks
    erlang:send_after(?MERGE_CHECK_INTERVAL, self(), {?MODULE, merge_check}),


riak_kv_multi_backend:

handle_info(State, Msg) ->
    F = fun(_Name, Module, SubState) ->
                Module:handle_info(SubState, Msg)
        end,    [F(X) || X <- State#state.backends],
    ok.

If riak_kv_multi_backend is configured in riak_kv_vnode the state's mod is riak_kv_multi_backend but the messages scheduled in riak_kv_bitcask_backend are going to have riak_kv_bitcask_backend as their Mod tag.

With my limited understanding about the rest of the sytem it seems to me that adding this case to riak_kv_vnode would fix the problem:
handle_info({Mod, Msg}, StateName, #state { mod = riak_kv_multi_backend } = StateData) ->
    riak_kv_multi_backend:handle_info(StateData#state.modstate, Msg),
    {next_state, StateName, StateData, ?TIMEOUT};

It is a bit wasteful because all the configured backends will get called with this message, but it is the best I can think of without leaking too much information about the specific backends into the generic code.

Modifying the handle_info case would work as well however one check would need to disappear:

handle_info({_ModTag, Msg}, StateName, #state { mod = Mod } = StateData) ->
    Mod:handle_info(StateData#state.modstate, Msg),
    {next_state, StateName, StateData, ?TIMEOUT}.

There are a plethora of other ways to fix this like passing the Mod tag to riak_kv_multi_backend so that it can filter based on it (this is probably my favourite as it is not wasteful and there aren't many code changes needed either):
riak_kv_vnode:
handle_info({Mod, Msg}, StateName, #state { mod = riak_kv_multi_backend } = StateData) ->
    riak_kv_multi_backend:handle_info(StateData#state.modstate, {Mod, Msg}),
    {next_state, StateName, StateData, ?TIMEOUT};

riak_kv_multi_backend:
handle_info(State, {Mod, Msg}) ->
    F = fun(_Name, Module, SubState) ->
                Module:handle_info(SubState, Msg)
        end,    [F(X) || X = {_, Module, _} <- State#state.backends, Module =:= Mod],
    ok.

Code is not tested, but should compile. :)

Regards,
    Tamas

-- 
Tamas Nagy
Erlang Solutions Ltd.
http://www.erlang-solutions.com
---------------------------------------------------

---------------------------------------------------

WE'VE CHANGED NAMES!

Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD.

www.erlang-solutions.com




More information about the riak-users mailing list