Planning JS VM capacity, and how the JS VM count is intended to work (preflist_exhausted, etc)
bconway at rcesoftware.com
Mon Apr 30 17:25:30 EDT 2012
There are no shortage of references out there to avoiding a
preflist_exhausted error due to JS VM capacity being too small when
My question is, are there any good guidelines for planning JS VM
capacity? For example, if you're running a particularly wide MapReduce
with JS and hit the "All VMs are busy" message in your logs, the
prescribed advice is to bump up the JS counts until you no longer hit
But isn't that just going to fail again as soon as you have more than
one M/R running at the same time?
Is it better to blindly increase the count into the thousands and hope
you cover all your bases, or set up a queuing mechanism outside of
Riak to fire off the MapReduce jobs at a rate that makes optimal use
of your hardware and JS VM count?
The MapReduce doc touches briefly on configuration tuning for
MapReduce phases wait for a free JS VM when all are in use, rather
than erroring out?
I may have a fundamental misunderstanding on how this is supposed to
work, my apologies if so. Thanks in advance.
More information about the riak-users