Tweaking app.config and vm.args for best performance

rsb cnasillo at gmail.com
Tue Jul 23 09:50:49 EDT 2013


Environment

- 5x Riak 1.4 nodes
- Bitcask as a backend
- Dedicated HDDs for each node storage, no RAID
- Virtualized environment (I know... I know... )

VM Host Specs

- 8 CPUs x 2.8 GHz (Xenon X5560)
- 65Gb of RAM
- Fiber optic network

Per Nodes Specs (Allocated Resources)

- Ubuntu server 12.04
- 11 Gb of RAM dedicated
- Dynamic CPU shares, should MHz be be reserved for each node?
- 7200 RPM, 320GB spindles 

The Data

Custom benchmarking application (Riak Java API using Protocol Buffers) where
each record is about 800 bytes in size. The application is currently
configured to simulate a client running 600 Threads, 10 batches per thread,
where each batch has 70 records. Therefore the total load is about 420k
records. We are looking to push in the future about 500 million items during
our benchmarks.

/Please feel free to suggest better ways to distribute the load; whether
that is more threads, smaller batches, etc.../

vm.args

Everything is pretty much on defaults, however I was wondering if I should
be tweaking any of the following;
- Kernel poll and async threads
- ERL_MAX_PORTS
- ERL_MAX_ETS_TABLES
- Erlang process limit

app.config

Same thing for the config file. Mostly default values. Any suggestions about
tweaking;
- disable_pb_nagle
- pb_backlog
- ring_creation_size
- map_js_vm_count
- reduce_js_vm_count
- hook_js_vm_count
- js_max_vm_mem
- js_thread_stack
- fsm_limit
- buffer_rollover_size
- gc_ms_limit
- heap_word_limit
- Any other missing setting that might not be present by default ?

OS Tuning

Apart from 'ulimit' which other settings should I be warry of that might
need setting? 

Bonus Question :)

We might be trying LevelDB as well, any I should be aware or model within my
application/settings to deal with this other backend?

Thank you a whole lot in advance! 



--
View this message in context: http://riak-users.197444.n3.nabble.com/Tweaking-app-config-and-vm-args-for-best-performance-tp4028510.html
Sent from the Riak Users mailing list archive at Nabble.com.




More information about the riak-users mailing list