Storage statistic calculation errors

Stanislav Vlasov stanislav.v.v at gmail.com
Fri Mar 13 00:39:46 EDT 2015


2015-03-13 6:45 GMT+05:00 Kazuhiro Suzuki <kaz at basho.com>:

> You can change the timeout for a MapReduce job the storage calculation
> uses. Could you try to add riakc section which contains mapred_timeout
> into Riak CS's app.config like this ? :

>  {riakc, [
>              {mapred_timeout, 1800000}],     %% msec
>      },
>

Current config contain:

 {riakc, [ %% increase timeout for LARGE bucket statistic calculation
         {mapred_timeout, 3600000}
         %% default mapred_call_timeout == 60000
         { mapred_call_timeout, 180000 }
 ]}

It's help to calculate without errors, but does not improve calculation speed.

>
> 2015-03-11 17:26 GMT+09:00 Stanislav Vlasov <stanislav.v.v at gmail.com>:
>> Our riak-cs cluster can't calculate storage statistic for some buckets
>> and all calculation takes too long:
>>
>> riak-cs/console.log:
>> 2015-03-11 01:25:56.791 [error]
>> <0.485.0>@riak_cs_storage:maybe_sum_bucket:75 failed to calculate
>> usage of bucket 'xxxxxxxxxxxxxxxxx' of user 'XXXXXXXXXXXXXXXXXXXX'.
>> Reason: {error,{timeout,[]}}
>> 2015-03-11 01:37:36.212 [info]
>> <0.485.0>@riak_cs_storage_d:calculating:150 Finished storage
>> calculation in 5794 seconds.
>>
>> This bucket contains over 5000000 files. It could be the cause of this
>> error, but several days ago, after cluster restart, all storage
>> statistic was calculated without errors:
>>
>> 2015-02-27 01:23:59.777 [info]
>> <0.483.0>@riak_cs_storage_d:calculating:150 Finished storage
>> calculation in 1138 seconds.
>>
>> Please advise anything to fix it
>>
>>
>> Our node config files and last logs can be found at http://ovh.to/MuavQVP
>>
>> 12 nodes in cluster, node hardware configuration:
>> CPU: 6 cores of Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
>> RAM: 60G
>>
>> --
>> Stanislav
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
> --
> Kazuhiro Suzuki | Basho Japan KK



-- 
Stanislav




More information about the riak-users mailing list