RIAK-CS crashing with many HEAD requests

Silasi Stefan silasistefan at gmail.com
Thu Apr 11 15:07:50 EDT 2013


Thanks Kelly. If you need test environment or more data, please let me know.

Thank you,
Stefan.


On Thu, Apr 11, 2013 at 8:57 PM, Kelly McLaughlin <kelly at basho.com> wrote:

> Stefan,
>
> I have been able to reproduce the problem you saw and I have opened a
> github issue to address it here:
> https://github.com/basho/riak_cs/issues/532. Thanks again for the report
> and we should have a fix for it pushed to github very soon.
>
> Kelly
>
>
> On Wed, Apr 10, 2013 at 5:42 PM, Kelly McLaughlin <kelly at basho.com> wrote:
>
>> Stefan,
>>
>> Thanks for the report. We will investigate this and let you know what we
>> find.
>>
>> Kelly
>>
>> On Wed, Apr 10, 2013 at 5:05 AM, Silasi Stefan <silasistefan at gmail.com>wrote:
>>
>>> Hello,
>>>
>>> I'm testing RIAK-CS as an S3 alternative and I get some strange crashing
>>> when reading data from it. The following is reproducible with each run of
>>> the python script.
>>>
>>> conn = boto.connect_s3(
>>>     aws_access_key_id = access_key,
>>>     aws_secret_access_key = secret_key,
>>>     host = riak_host,
>>>     proxy = riak_host,
>>>     proxy_port = 8080,
>>>     calling_format = boto.s3.connection.OrdinaryCallingFormat(),
>>>     debug = 2,
>>>     is_secure = False,
>>> )
>>>
>>> bucket = conn.create_bucket('bucket_name')
>>>
>>> for row in rows:
>>>     key=bucket.new_key(valid_key_name)
>>>     if key.exists():
>>>         key.open_read()
>>>
>>>
>>> If I monitor the riak-cs memory usage, it is growing and growing until
>>> it's using all RAM+SWAP, then it crashes.
>>> In the console log, the following line is appended before RIAK-CS
>>> crashes:
>>>
>>> 2013-04-10 08:28:40.802 [info] <0.41.0> alarm_handler:
>>> {set,{system_memory_high_watermark,[]}}
>>>
>>> Nothing related in the error.log.
>>> In access.log I see a lot of HEAD requests for the objects stored.
>>>
>>> Current architecture looks like this:
>>> - 3 virtual machines with 4 cores, 4gb ram, 2gb swap and 1tb disk each
>>> - RIAK installed on all of them, configured as cluster
>>> - stanchion installed on all of them
>>> - RIAK-CS installed on all of them, configured to "speak" only with
>>> local RIAK
>>> - about 10GB of data storred in each RIAK with N=3.
>>>
>>> The strange thing is that if I move the connect-to part in the FOR loop,
>>> RIAK-CS memory usage doesn't go over 300MB. Any ideas what could be wrong?
>>>
>>> Thank you,
>>> Stefan.
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> riak-users at lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130411/a7065489/attachment.html>


More information about the riak-users mailing list