Riak memory usage higher than expected

Nico Meyer nico.meyer at adition.com
Wed Aug 17 03:07:52 EDT 2011


Hi Jacques,

please read my reply back in May, which should clear things up:

http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-May/004292.html

Cheers,
Nico

On 16.08.2011 21:09, Jacques wrote:
> We're utilizing Riak 14.2 and we're seeing higher memory consumption
> than we expect.
>
> We're running on a 4 node cluster with each node housing 32gb of memory
> and are utilizing bitcask with a 3x write replication factor.  We're
> seeing faster growth than we expect and also seeing weird bounces update.
>
> You can see an example chart <http://picturepush.com/public/6331935>.
>   (note, there are sometimes where we've had to stop the job for short
> periods of time-- you can see these as flat spots).
>
> We are doing a large throttled import that has key sizes of
> approximately 12 bytes.  We're currently around 450mm unique items and
> riak memory consumption is ~110gb.  The input job is probably 95% new
> puts and 5% overwriting puts.
>
> According to the capacity planner tools, our key space should probably
> be about half what are actual memory consumption is.
>
> As you can see in the chart, we're also seeing jumps in memory size at
> random intervals.  What might these be?  Nothing interesting in the logs
> that I can see.  Regular merges.
>
> A close up <http://picturepush.com/public/6332084> of a recent jump in
> memory consumption for one of the nodes (they all look the same).  There
> are no corresponding distinct patterns within the cpu chart.  Things are
> pretty flat although we have more wait sometimes than we like (need more
> spindles clearly).
>
>
> Any helpful thoughts?
>
> Thanks,
> Jacques
>
>
>
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com




More information about the riak-users mailing list