Reg:Continuous Periodic crashes after long operation

Steven Joseph steven at streethawk.com
Fri Jan 27 08:42:57 EST 2017


I've had this issue again, this time I checked the output of lsof and it
seems like its the number of established connections are way high, I've
configured my application tasks to exit and cleanup connections
periodicaly. That should solve it.

Thanks guys.

Steven

On Fri, Jan 27, 2017 at 3:07 AM Matthew Von-Maszewski <matthewv at basho.com>
wrote:

> FYI:  this is the function that is crashing:
>
> get_uint32_measurement(Request, #internal{os_type = {unix, linux}}) ->
>     {ok,F} = file:open("/proc/loadavg",[read,raw]),                  %%
> <--- crash line
>     {ok,D} = file:read(F,24),
>     ok = file:close(F),
>     {ok,[Load1,Load5,Load15,_PRun,PTotal],_} = io_lib:fread("~f ~f ~f
> ~d/~d", D),
>     case Request of
>         ?avg1  -> sunify(Load1);
>         ?avg5  -> sunify(Load5);
>         ?avg15 -> sunify(Load15);
>         ?ping -> 4711;
>         ?nprocs -> PTotal
>     end;
>
> Is there something unique about that open?
>
> Matthew
>
> > On Jan 26, 2017, at 10:37 AM, Luke Bakken <lbakken at basho.com> wrote:
> >
> > Steven,
> >
> > You may be able to get information via the lsof command as to what
> > process(es) are using many file handles (if that is the cause).
> >
> > I searched for that particular error and found this GH issue:
> > https://github.com/emqtt/emqttd/issues/426
> >
> > Which directed me to this page:
> > https://github.com/emqtt/emqttd/wiki/linux-kernel-tuning
> >
> > Basho also has a set of recommended tuning parameters:
> > http://docs.basho.com/riak/kv/2.2.0/using/performance/
> >
> > Do you have other error entries in any of Riak's logs at around the
> > same time as these messages? Particularly crash.log.
> >
> > --
> > Luke Bakken
> > Engineer
> > lbakken at basho.com
> >
> > On Thu, Jan 26, 2017 at 4:42 AM, Steven Joseph <steven at streethawk.com>
> wrote:
> >> Hi Shaun,
> >>
> >> I have already set this to a very high value
> >>
> >> (riak at hawk1.streethawk.com)1> os:cmd("ulimit -n").
> >> "20000500\n"
> >> (riak at hawk1.streethawk.com)2>
> >>
> >>
> >> So the issue is not that the limit is low, but maybe a resource leak ?
> As I
> >> mentioned our application processes continuously run queries on the
> cluster.
> >>
> >> Kind Regards
> >>
> >> Steven
> >
> > _______________________________________________
> > riak-users mailing list
> > riak-users at lists.basho.com
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20170127/4c3f4281/attachment-0002.html>


More information about the riak-users mailing list