riak-cs reclaim storage

Kota Uenishi kota at basho.com
Sun Jun 8 21:10:26 EDT 2014


Hi,

Are you using 1.4.5? I'm not sure but you may have a log message like this:

"Error occurred trying to query from time 0 to ~p in gc key index. Reason: ~p"

in console.log. If so, it's even failing not only GC but in listing
objects to be collected. Then setting a line

  {gc_pagenated_indexes, true},

into riak_cs section of your app.config . This is implicit option and
its default is false.


On Fri, Jun 6, 2014 at 11:52 PM, Tyler Flint <tylerflint at gmail.com> wrote:
> Thank you in advance for any assistance.
>
> To explore riak-cs, I setup a 2 node cluster with riak and riak-cs on each node, and put the riak nodes on the same cluster. I then proceeded to use risk-cs by uploading a 3G file using the multi-part upload protocol. I was unaware, however, that I needed to finalize the multipart upload and thus had many in progress multi-part uploads. Through the duration of my troubleshooting, the storage on the filesystem grew to 224G. Once I discovered my error, I proceeded to cancel all of the previous uploads. At this point there are no uploads in progress, and only a single 3G file available in the bucket. However, it has been 48 hours since the exploration and the storage use is still 224G on both machines.
>
> I have attempted multiple strategies in attempt to reclaim space, all without any effect.
>
> Within riak-cs:
>
> - I have shortened the gc leeway duration to 5 minutes and set all of the gc intervals as low as 5 minutes.
> - I have manually triggered the riak-cs-gc batch multiple times. (After running batch, status shows a gc run and eventually complete)
>
> Within riak:
>
> - I have adjusted the bitcask settings in attempt to trigger a merge.
> - I have force merged all of the bitcask partitions manually in the erlang console.
>
> None of these strategies seem to have any effect whatsoever.
>
> At this point I’m not even sure where to start, nor do I know if the issue is within riak-cs or riak (via bitcask). Any suggestions would be appreciated.
>
> Thanks
>
> Tyler
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



-- 
Kota UENISHI / @kuenishi
Basho Japan KK




More information about the riak-users mailing list