General Memory/Performance Tuning Guidelines

Gordon Tillman gtillman at
Mon May 23 14:05:08 EDT 2011


We are working with a riaksearch cluster that uses innostore as the primary backend in tandem with merge_index that is required by search.  From reading the Basho wiki it looks like the following are the most important factors affecting memory and performance:

	• innostore
		• put data_home_dir and log_group_home_dir on different spindles
		• noatime
		• buffer_pool_size
		• flush_method
	• merge_index
		• data_root
		• buffer_rollover_size
		• max_compact_segments
		• segment_file_buffer_size
		• segment_full_read_size
		• segment_block_size

Ideally, data_home_dir, log_group_home_dir, and data_root would all be on different spindles, but if you had just 2 disks available what would you recommend?  Would it be best to have data_home_dir and data_root on one and then log_group_home_dir on the other?

in calculating the proper setting for buffer_pool_size you are directed to allocate 60-80 percent of available RAM.  So lets assume you want to take the remaining 20-40% of available RAM and split it up between innostore and merge_index?  

Would it be best to give each of them half of that value?

Determining the approximate memory requirements for merge_index isn't (to me) real obvious.  I looks like the following all have  an effect:

 * buffer_rollover_size
 * buffer_delayed_write_size
 * max_compact_segments
 * segment_query_read_ahead_size
 * segment_compaction_read_ahead_size
 * segment_full_read_size
 * segment_block_size
 * segment_values_staging_size

Is there a formula for determining the (approximate) proper values to use given a certain amount of available RAM?

Thanks in advance for any advice.  Sorry for all the questions!


More information about the riak-users mailing list