crash after single insert

Eric Moritz eric at themoritzfamily.com
Sat May 7 20:15:24 EDT 2011


Dang, that was my guess but I wasn't sure if it was a red herring or not.

On Sat, May 7, 2011 at 11:25 AM, Gary William Flake <gary at flake.org> wrote:
> That was it, Nico.  Thanks.
>
> I know we did a forced shutdown this week, which was probably the cause.  But I would have thought that riak would have taken care of its own lock file bookkeeping on restarting.
>
>
>
>
> On May 7, 2011, at 12:45 AM, Nico Meyer wrote:
>
>> Hi Gary,
>>
>> I had this problem before a few times.
>> It is cause by leftover files named 'bitcask.write.lock'. Normally theey should be cleared automatically after a crash. They contain the OS process ID of the erlang VM which created them, and if there is no such process bitcask deletes the file.
>> I believe that there might exists another process with the same process ID as the old riak process. Did you reboot the machine by any chance? since then the PIDs start counting up from 1 again.
>>
>> The easy fix for your problem is to delete all bitcask.write.lock files in your bitcask directory (e.g. 'rm /var/lib/bitcask/*/bitcask.write.lock').
>> If you are curious you could check if my hunch is right beforehand. Just look at the contents of any of these files (which looks like: PID /path/to/current/bitcaskfile/123456.bitcask.data), and see if there is another process with the PID.
>>
>> Cheers Nico
>>
>>
>> On 06.05.2011 23:08, Gary William Flake wrote:
>>> Now trying 500 lines.
>>> -- GWF
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On May 6, 2011, at 2:05 PM, Gary William Flake wrote:
>>>
>>>> Resending with only the last 1000 lines from the log file, so that the list server doesn't reject it.
>>>>
>>>> Any insights would be greatly appreciated.
>>>>
>>>> -- GWF
>>>>
>>>> <tail-sasl-error.log>
>>>>
>>>>
>>>> On May 6, 2011, at 1:48 PM, Gary William Flake wrote:
>>>>
>>>>> Today, after running fine for a couple of weeks, I had erlang crash, making it so that riaksearch can only do reads but no writes.
>>>>>
>>>>> Nothing has changed with the code and it had been running fine for weeks without a problem.  The sasl-error.log is attached.
>>>>>
>>>>> Any ideas what's going on?
>>>>>
>>>>> Thanks,
>>>>> -- GWF
>>>>>
>>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> riak-users at lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>




More information about the riak-users mailing list