Riak CS: Undeletable broken buckets

Toby Corkindale toby at dryft.net
Mon Jul 7 01:12:30 EDT 2014


Hi Andrew,
Thanks for the details.
The Puppet config should never have let it be setup with
allow_mult=false, but as this is a test cluster, it's possible
something went awry there at some point.

If it's not really a bug that needs reporting then I can let it go.
Thanks,,
Toby

On 7 July 2014 12:21, Andrew Stone <astone at basho.com> wrote:
> Hi Toby,
>
> We've seen this scenario before. It occurs because riak-cs stores bucket
> information in 2 places on disk:
>   1) Inside the user record (for bucket permissions)
>   2) Inside a global list of buckets, since each bucket must be unique
>
> What has happened most likely is that the bucket is no longer stored for the
> given user, but still in the global list of bucket. It shows up in bucket
> lists, but the current user doesn't have permission to actually do anything
> with it. Essentially you have partially written (or partially deleted) data.
> I believe the only time we saw this was when Riak was configured with
> {allow_mult, false} which is an invalid setting when used with riak-cs.
> Riak-cs uses siblings intelligently to merge conflicting data, and without
> that it's possible to end up in these types of scenarios. Later versions of
> riak-cs should refuse to run with {allow_mult, false}. I'd check your riak
> config to see if that is the case here.
>
> We actually have scripts to detect and remove the bad buckets that we've
> used in support. We can probably get you a copy if you want. Just let me
> know. And make sure when running in production that allow_mult = true.
>
> -Andrew
>
>
>
> On Sun, Jul 6, 2014 at 9:59 PM, Toby Corkindale <toby at dryft.net> wrote:
>>
>> Hi,
>> At some point we've managed to create a couple of buckets that don't
>> work and can't be deleted (in a development/testing cluster, not
>> production).
>> They show up with both 's3cmd ls' or by querying the HTTP API for a
>> user's buckets.
>> However attempting to list files in the bucket, or removing the
>> bucket, or recreating the bucket, fails.
>>
>> It's not in a production cluster so it's not a huge concern to me, but
>> thought I'd report the bug here in case it's of interest to you.
>> Riak 1.4.9-1 and Riak-CS 1.4.5-1 on Ubuntu 12.04 LTS.
>>
>> $ s3cmd ls
>> 2014-02-07 00:07  s3://test5403
>> 2013-12-13 07:25  s3://test9857
>>
>> $ s3cmd ls s3://test5403
>> ERROR: Bucket 'test5403' does not exist
>> tobyc at adonai:~$ s3cmd ls s3://test9857
>> ERROR: Bucket 'test9857' does not exist
>>
>> $ s3cmd rb s3://test5403
>> ERROR: Bucket 'test5403' does not exist
>> Bucket 's3://test5403/' removed
>>
>> $ s3cmd ls
>> 2014-02-07 00:07  s3://test5403
>> 2013-12-13 07:25  s3://test9857
>>
>> $ s3cmd mb s3://test5403
>> Bucket 's3://test5403/' created
>>
>> $ s3cmd ls s3://test5403
>> ERROR: Bucket 'test5403' does not exist
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>



-- 
Turning and turning in the widening gyre
The falcon cannot hear the falconer
Things fall apart; the center cannot hold
Mere anarchy is loosed upon the world




More information about the riak-users mailing list