Search precommit hook bug?

Elias Levy fearsome.lucidity at gmail.com
Wed Sep 21 12:26:10 EDT 2011


On Wed, Sep 21, 2011 at 6:32 AM, David Smith <dizzyd at basho.com> wrote:

> On Tue, Sep 20, 2011 at 12:41 PM, Elias Levy
> <fearsome.lucidity at gmail.com> wrote:
>
> > Now from what I've been able to find,{error,emfile} usually means you are
> > out of file descriptors.  Yes?
>
> Yes. If you are running a default configuration, however, you will
> need considerably more than 4096 file handles.


So it seems.  Tried again yesterday setting the limit to 10K.  It almost
finish loading my 6 million objects.  So even 10K was not sufficient.  At
the moment the riak process in one of the nodes shows 8,814 open fds while
idle.

You can tighten this
> down by setting:
>
> {max_open_files, 10}
>
> in the eleveldb section of app.config. That will yield 10 open file
> handles _per_ vnode -- on a out of the box setup that's 64 * 10 file
> handles necessary.
>

Thanks.  It makes sense now.  I suppose all this will be documented once 1.0
is released.  You may want to change the default value in the eleveldb
backend.  Probably not a good idea to have a default that can result on the
system starving for file handles.

Any idea what kind of performance hit one takes by lowering the number of
open file handles?

We are using a ring size of 128.  In a worse case scenario all those 128
vnodes would exist on a single node, so we need 128*max_open_files fd limit
per node.

What about replicas?  Do they have their own vnode?  Or does a ring with a
size of 128 can only have at most 128 vnodes regardless of replicas or
fallbacks?

Elias Levy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110921/6209ba6c/attachment.html>


More information about the riak-users mailing list