Reg:Continuous Periodic crashes after long operation

Luke Bakken lbakken at basho.com
Wed Feb 1 17:15:37 EST 2017


Thanks for the information. Yes, one RiakClient instance per Unix
process is correct.

I will see if there is a way for you to keep track of connections from
the client to Riak. Off the top of my head the Python client doesn't
have the ability to set limits.

--
Luke Bakken
Engineer
lbakken at basho.com

On Wed, Feb 1, 2017 at 1:59 PM, Steven Joseph <steven at streethawk.com> wrote:
> Hi Luke,
>
> Yes I am creating new client objects for each of my tasks.
>
> Please see this github issuse against the python client for some
> background as to why.
>
> https://github.com/basho/riak-python-client/issues/497
>
> Basicaly I ran into issues with concurrency when processes are forked.
>
> I might experiment with using process ids as keys to access a process
> specific riak client in forked child ?
>
>
> Regards
>
> Steven
>
> Luke Bakken <lbakken at basho.com> writes:
>
>> Hi Steven,
>>
>> At this point I suspect you're using the Python client in such a way
>> that too many connections are being created. Are you re-using the
>> RiakClient object or repeatedly creating new ones? Can you provide any
>> code that reproduces your issue?
>>
>> --
>> Luke Bakken
>> Engineer
>> lbakken at basho.com
>>
>>
>> On Tue, Jan 31, 2017 at 7:47 PM, Steven Joseph <steven at streethawk.com> wrote:
>>> Hi Luke,
>>>
>>> Here's the output of
>>>
>>> $ sysctl fs.file-max
>>>
>>> fs.file-max = 20000500
>>>
>>> Regards
>>>
>>> Steven
>>>
>>> On Wed, Feb 1, 2017 at 9:30 AM Luke Bakken <lbakken at basho.com> wrote:
>>>>
>>>> Hi Steven,
>>>>
>>>> What is the output of this command on your systems?
>>>>
>>>> $ sysctl fs.file-max
>>>>
>>>> Mine is:
>>>>
>>>> fs.file-max = 1620211
>>>>
>>>> --
>>>> Luke Bakken
>>>> Engineer
>>>> lbakken at basho.com
>>>>
>>>>
>>>> On Tue, Jan 31, 2017 at 12:22 PM, Steven Joseph <steven at streethawk.com>
>>>> wrote:
>>>> > Hi Shaun,
>>>> >
>>>> > Im having this issue again, this time I have captured the system limits,
>>>> > while riak is still crashing.
>>>> >
>>>> > Please note lsof and prlimit outputs at bottom.




More information about the riak-users mailing list