Storage statistic calculation errors

Kota Uenishi kota at basho.com
Thu Mar 12 22:08:24 EDT 2015


Which version of Riak CS are you using? If it's 1.5.3 or later,
"storage_calc_timeout" in "riak_cs" section should be used instead of
"mapred_timeout" because it won't work [1]. This is undocumented, but
we should've documented this.

Moreover, in CS 1.5.3 or later "mapred_timeout" or "timeout" in
"riakc" section won't work due to the same reason - all names of
configuration knobs can be found in code [2]. I hope if you don't read
Erlang, this code might be simple enough to understand what's
configurable.

[1] https://github.com/basho/riak_cs/blob/release/1.5/RELEASE-NOTES.md#additions
[2] https://github.com/basho/riak_cs/blob/release/1.5/src/riak_cs_config.erl#L418

On Fri, Mar 13, 2015 at 10:45 AM, Kazuhiro Suzuki <kaz at basho.com> wrote:
> Hi Stanislav,
>
> You can change the timeout for a MapReduce job the storage calculation
> uses. Could you try to add riakc section which contains mapred_timeout
> into Riak CS's app.config like this ? :
>
> ```
> [
> %% riakc section
>  {riakc, [
>              {mapred_timeout, 1800000}],     %% msec
>      },
>
> [
>  %% Riak CS section
>  {riak_cs, [
>
>
> ```
>
> Thanks,
> Kaz
>
>
> 2015-03-11 17:26 GMT+09:00 Stanislav Vlasov <stanislav.v.v at gmail.com>:
>> Our riak-cs cluster can't calculate storage statistic for some buckets
>> and all calculation takes too long:
>>
>> riak-cs/console.log:
>> 2015-03-11 01:25:56.791 [error]
>> <0.485.0>@riak_cs_storage:maybe_sum_bucket:75 failed to calculate
>> usage of bucket 'xxxxxxxxxxxxxxxxx' of user 'XXXXXXXXXXXXXXXXXXXX'.
>> Reason: {error,{timeout,[]}}
>> 2015-03-11 01:37:36.212 [info]
>> <0.485.0>@riak_cs_storage_d:calculating:150 Finished storage
>> calculation in 5794 seconds.
>>
>> This bucket contains over 5000000 files. It could be the cause of this
>> error, but several days ago, after cluster restart, all storage
>> statistic was calculated without errors:
>>
>> 2015-02-27 01:23:59.777 [info]
>> <0.483.0>@riak_cs_storage_d:calculating:150 Finished storage
>> calculation in 1138 seconds.
>>
>> Please advise anything to fix it
>>
>>
>> Our node config files and last logs can be found at http://ovh.to/MuavQVP
>>
>> 12 nodes in cluster, node hardware configuration:
>> CPU: 6 cores of Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
>> RAM: 60G
>>
>> --
>> Stanislav
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
> --
> Kazuhiro Suzuki | Basho Japan KK
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



-- 
Kota UENISHI / @kuenishi
Basho Japan KK




More information about the riak-users mailing list