Riak consumes too much memory

darren darren at ontrenet.com
Fri Oct 18 14:08:12 EDT 2013


Is it smart enough to manage itself?
Or does it require human babysitting?


Sent from my Verizon Wireless 4G LTE Smartphone

-------- Original message --------
From: Matthew Von-Maszewski <matthewv at basho.com> 
Date: 10/18/2013  1:48 PM  (GMT-05:00) 
To: Dave Martorana <dave at flyclops.com> 
Cc: darren <darren at ontrenet.com>,riak-users at lists.basho.com 
Subject: Re: Riak consumes too much memory 
 
Dave,

flexcache will be a new feature in Riak 2.0.  There are some subscribers to this mailing list that like to download and try things early.  I was directing those subscribers to the GitHub branch that contains the work-in-progress code.

flexcache is a new method for sizing / accounting the memory used by leveldb.  It replaces the current method completely.  flexcache is therefore not an option, but an upgrade to the existing logic.

Again, the detailed discussion is here:  ttps://github.com/basho/leveldb/wiki/mv-flexcache

Matthew


On Oct 18, 2013, at 12:33 PM, Dave Martorana <dave at flyclops.com> wrote:

Matthew,

For we who don't quite understand, can you explain - does this mean mv-flexcache is a feature that just comes with 2.0, or is it something that will need to be turned on, etc?

Thanks!

Dave


On Thu, Oct 17, 2013 at 9:45 PM, Matthew Von-Maszewski <matthewv at basho.com> wrote:
It is already in test and available for your download now:

https://github.com/basho/leveldb/tree/mv-flexcache

Discussion is here:

https://github.com/basho/leveldb/wiki/mv-flexcache

This code is slated for Riak 2.0.  Enjoy!!

Matthew

On Oct 17, 2013, at 20:50, darren <darren at ontrenet.com> wrote:

But why isn't riak smart enough to adjust itself to the available memory or lack thereof?

No serious enterprise technology should just consume everything and crash.


Sent from my Verizon Wireless 4G LTE Smartphone



-------- Original message --------
From: Matthew Von-Maszewski <matthewv at basho.com> 
Date: 10/17/2013 8:38 PM (GMT-05:00) 
To: ZhouJianhua <jh.zhou at outlook.com> 
Cc: riak-users at lists.basho.com 
Subject: Re: Riak consumes too much memory 


Greetings,

The default config targets 5 servers and 16 to 32G of RAM.  Yes, the app.config needs some adjustment to achieve happiness for you:

- change ring_creation_size from 64 to 16 (remove the % from the beginning of the line)
- add this line before "{data_root, <path>}" in eleveldb section: "{max_open_files, 40}," (be sure the comma is at the end of this line).

Good luck,
Matthew


On Oct 17, 2013, at 8:23 PM, ZhouJianhua <jh.zhou at outlook.com> wrote:

Hi

I installed riak v1.4.2 on ubuntu12.04(64bit, 4G RAM) with apt-get, run it with default app.conf but change the backend to leveldb, and test it with https://github.com/tpjg/goriakpbc .

Just keep putting (key, value) to an bucket, the memory always increasing, and in the end it crashed, as it cannot allocate memory.

Should i change the configuration or other?
_______________________________________________
riak-users mailing list
riak-users at lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
riak-users at lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20131018/ca43831d/attachment.html>


More information about the riak-users mailing list