2i performance questions

Tom Burdick thomas.burdick at gmail.com
Thu Jun 14 15:29:07 EDT 2012


Hi, I'm currently using 2i and riak to store oauth authorization grants.

Currently my store function (using riak-erlang-client) is basically,

store(Db, Grant) ->
    Key = token(Grant),
    Value = term_to_binary(Grant),
    Indexes = [
        {<<"expires_int">>, expires(Grant)},
        {<<"client_id_bin">>, client_id(Grant)},
        {<<"user_id_bin">>, user_id(Grant)},
        {<<"refresh_token_bin">>, refresh_token(Grant)}
    ],
    RObject = riakc_obj:new(?bucket, Key, Value,
<<"application/x-erlang-term">>),
    MetaData = dict:store(?MD_INDEX, Indexes, dict:new()),
    RObject0 = riakc_obj:update_metadata(RObject, MetaData),
    riakc_pb_socket:put(Db, RObject0).

Later I'd like to query on any of the indexes, so something like...

retrieve_by_client(Db, Key) when is_binary(Key) ->
    {ok, BKeys} = riakc_pb_socket:get_index(Db, ?bucket,
<<"client_id_bin">>, Key),
    lists:map(fun([_Bucket, Key0]) ->
        Key0
    end, BKeys).

What I've noticed while benchmarking the put is that that is actually
quite fast and I can do that
at a pretty high rate even on my devrel setup, like 1000 puts/sec.

However when doing retrieve_by_client at most I've seen 20
get_indexes/sec with latencies all over the place from a few
milliseconds up to a few seconds.

Notably when benchmarking get_index thats all thats going on, there
are no writes or other reads. I've also tried adjust the number of
concurrent benchmark workers and saw basically no change in latency or
get_index rate.

What have other people experienced while using 2i?

I recently watched the videos from kiip.me mentioning they found 2i to
also mention having high latency and a boundary video where they too
mention using 2i sparingly because of issues.

Are there any tweaks to riak's config that might dramatically change
my results?

The config is a default devrel with leveldb setup as the backend
instead of bitcask, thats it.

I'm tempted to fire up fprof on Client:get_index just to see whats up
but I'm a bit scared at what I might see there :-)

Thanks!

Tom Burdick




More information about the riak-users mailing list