Riak taking up lots memory on a write heavy bucket?

Metin Akat akat.metin at gmail.com
Fri Aug 24 05:11:27 EDT 2012

Hi Mark,

Our dev cluster is:
3x  1 core, 1G ram x86 ubuntu 12.04  VMs
plus 2x 2core 4G ram x64 ubuntu 12.04 VMs

I know, this is probably suboptimal, but it's only used for development.
So far it seems that the memory usage has auto balanced around 500-600M per
node (regardless of server's available memory)
Before introducing the functionality with the bucket in question, all nodes
used < 100M RAM. This is still the case on our production cluster (which is
not yet merged). As far as I see, there isn't any big problem for us
merging this, as 500-600M is fine, especially as we are not experiencing
any problems like loss of data or whatever.

Nevertheless, I send you the cluster-info output file, if you are
interested to investigate it.

On Thu, Aug 23, 2012 at 1:41 AM, Mark Phillips <mark at basho.com> wrote:

> Sorry for the late response here, Metin.
> On Thu, Jul 26, 2012 at 1:08 AM, Metin Akat <akat.metin at gmail.com> wrote:
> > Hi list,
> >
> > I have a bucket with allow_mult=false, last_write_wins=true.  I use it to
> > store text files (up to 1 megabyte of text). Some of these objects are
> write
> > heavy. The application overwrites them several times a minute (at times).
> > Now I see my dev riak node (where I'm still developing the feature) is
> > taking up 450M of reserved memory. Is this normal? is this caused by the
> > bucket/objects in question? Is it dangerous, will it start eating even
> more
> > memory, or will it auto balance itself (seems likely for now)? How can I
> > diagnose what is taking up this memory? Riaknostic doesn't show anything
> of
> > particular interest.
> > Also, what are the correct settings to be used for such kind of buckets?
> > It's storing data of no high value (n_val=1) which is autoexpiring
> > (bitcask). It is also indexed in riak search. (btw, is the search index
> for
> > an object auto deleted when that object expires?).
> >
> So, as far as the memory usage goes, it's hard to say. If you were
> using vector clocks, I would say you were creating a huge object via
> crazy sibling generation. That said, with lww enabled, it's hard to
> say.  A few questions:
> * What size machines are in your dev cluster? How many?
> * The "riak-admin cluster_info" utility [0] might be useful here. Can
> you run that and send along the results?
> Mark
> [0] http://wiki.basho.com/Command-Line-Tools---riak-admin.htm (towards
> the bottom)
> > _______________________________________________
> > riak-users mailing list
> > riak-users at lists.basho.com
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20120824/c459410a/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cluster-info.txt.gz
Type: application/x-gzip
Size: 956329 bytes
Desc: not available
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20120824/c459410a/attachment.gz>

More information about the riak-users mailing list