Riak memory usage higher than expected
nico.meyer at adition.com
Thu Aug 18 06:31:06 EDT 2011
Have you taken into account the size of the bucket name? is it 12 bytes
Also there is some additional overhead that is due to memory alignment
requirements, depending on your CPU architecture.
I have done some tests on this for x86_64 a while ago:
Not much you can do about that, unless you want to use unaligned memory,
which is super complicated (replace malloc with your own allocator) and
has a huge performance penalty.
So lets assume your 12 bytes are include bucket and key:
43 byte overhead + 12 byte keydata = 55 bytes, this is the number of
bytes allocated per entry. In reality malloc(55) allocates 64 bytes
internally, so you total memory consumption is:
64byte*3*450Mio = 80 Gb
But if the bucket size is not included in the 12 bytes, and your bucket
name(s) has at least a size of 2 bytes we get:
(43 + 12 + 2) bytes = 57 bytes, and in this case malloc(57) really
allocates 80 bytes inernally. Therefore:
80byte*3*450Mio = 100Gb
Am 17.08.2011 19:43, schrieb Jacques:
> So if my math is correct, consumption should be around 70-80 for me.
> Yet we see a usage over 100. I'm also notclear what the step changes
> are. If you have any insights on these that would be great. Thanks!
> On Aug 17, 2011 12:07 AM, "Nico Meyer" <nico.meyer at adition.com
> <mailto:nico.meyer at adition.com>> wrote:
> > Hi Jacques,
> > please read my reply back in May, which should clear things up:
> > Cheers,
> > Nico
> > On 16.08.2011 21:09, Jacques wrote:
> >> We're utilizing Riak 14.2 and we're seeing higher memory consumption
> >> than we expect.
> >> We're running on a 4 node cluster with each node housing 32gb of memory
> >> and are utilizing bitcask with a 3x write replication factor. We're
> >> seeing faster growth than we expect and also seeing weird bounces
> >> You can see an example chart <http://picturepush.com/public/6331935>.
> >> (note, there are sometimes where we've had to stop the job for short
> >> periods of time-- you can see these as flat spots).
> >> We are doing a large throttled import that has key sizes of
> >> approximately 12 bytes. We're currently around 450mm unique items and
> >> riak memory consumption is ~110gb. The input job is probably 95% new
> >> puts and 5% overwriting puts.
> >> According to the capacity planner tools, our key space should probably
> >> be about half what are actual memory consumption is.
> >> As you can see in the chart, we're also seeing jumps in memory size at
> >> random intervals. What might these be? Nothing interesting in the logs
> >> that I can see. Regular merges.
> >> A close up <http://picturepush.com/public/6332084> of a recent jump in
> >> memory consumption for one of the nodes (they all look the same). There
> >> are no corresponding distinct patterns within the cpu chart. Things are
> >> pretty flat although we have more wait sometimes than we like (need
> >> spindles clearly).
> >> Any helpful thoughts?
> >> Thanks,
> >> Jacques
> >> _______________________________________________
> >> riak-users mailing list
> >> riak-users at lists.basho.com <mailto:riak-users at lists.basho.com>
> >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the riak-users