RIAK-CS crashing with many HEAD requests

Kelly McLaughlin kelly at basho.com
Thu Apr 11 14:57:51 EDT 2013


Stefan,

I have been able to reproduce the problem you saw and I have opened a
github issue to address it here: https://github.com/basho/riak_cs/issues/532.
Thanks again for the report and we should have a fix for it pushed to
github very soon.

Kelly

On Wed, Apr 10, 2013 at 5:42 PM, Kelly McLaughlin <kelly at basho.com> wrote:

> Stefan,
>
> Thanks for the report. We will investigate this and let you know what we
> find.
>
> Kelly
>
> On Wed, Apr 10, 2013 at 5:05 AM, Silasi Stefan <silasistefan at gmail.com>wrote:
>
>> Hello,
>>
>> I'm testing RIAK-CS as an S3 alternative and I get some strange crashing
>> when reading data from it. The following is reproducible with each run of
>> the python script.
>>
>> conn = boto.connect_s3(
>>     aws_access_key_id = access_key,
>>     aws_secret_access_key = secret_key,
>>     host = riak_host,
>>     proxy = riak_host,
>>     proxy_port = 8080,
>>     calling_format = boto.s3.connection.OrdinaryCallingFormat(),
>>     debug = 2,
>>     is_secure = False,
>> )
>>
>> bucket = conn.create_bucket('bucket_name')
>>
>> for row in rows:
>>     key=bucket.new_key(valid_key_name)
>>     if key.exists():
>>         key.open_read()
>>
>>
>> If I monitor the riak-cs memory usage, it is growing and growing until
>> it's using all RAM+SWAP, then it crashes.
>> In the console log, the following line is appended before RIAK-CS crashes:
>>
>> 2013-04-10 08:28:40.802 [info] <0.41.0> alarm_handler:
>> {set,{system_memory_high_watermark,[]}}
>>
>> Nothing related in the error.log.
>> In access.log I see a lot of HEAD requests for the objects stored.
>>
>> Current architecture looks like this:
>> - 3 virtual machines with 4 cores, 4gb ram, 2gb swap and 1tb disk each
>> - RIAK installed on all of them, configured as cluster
>> - stanchion installed on all of them
>> - RIAK-CS installed on all of them, configured to "speak" only with local
>> RIAK
>> - about 10GB of data storred in each RIAK with N=3.
>>
>> The strange thing is that if I move the connect-to part in the FOR loop,
>> RIAK-CS memory usage doesn't go over 300MB. Any ideas what could be wrong?
>>
>> Thank you,
>> Stefan.
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130411/d520ab8c/attachment.html>


More information about the riak-users mailing list