Problem with deleting keys

james at populi.co james at populi.co
Fri Aug 15 16:48:50 EDT 2014


Old thread, but we're still running into similar issues in 2014 (possibly due
to https://github.com/basho/riak_kv/issues/311)

In case it helps anyone, here's a quick ruby script to crawl through and
remove the tombstones 

*use at your own risk* - you're not supposed to list all keys in production,
but for us the alternative was to have 560K tombstones sitting around taking
up space when we only needed ~3K active keys.  Toss this in a monthly cron
job and there you go - also, if you needed the script to use less memory you
could alter the first curl request to: `curl
'localhost:8098/buckets/YOUR_BUCKET_HERE/keys?keys=stream >
/tmp/tmp_file.json` and then parse the resulting JSON file one chunk at a
time.

Note that the ?pw=all&pr=all&w=all&r=all is necessary (as per this thread).


riak_tombstone_cleanup.rb 
-----------------

require 'json'

puts "Getting list of riak keys"
keys = JSON.parse(`curl
'localhost:8098/buckets/YOUR_BUCKET_HERE/keys?keys=true'`)['keys']

puts "#{keys.count} keys loaded"

bad_keys_counter = 0
good_keys_counter = 0
keys.each do |key|
  result = `curl -I
'localhost:8098/buckets/YOUR_BUCKET_HERE/keys/#{key}?pw=all&pr=all&w=all&r=all'
2>&1`

  if result['404 Object Not Found']
    bad_keys_counter += 1
  else
    good_keys_counter += 1
  end
  puts "Processed #{bad_keys_counter} bad key(s) and #{good_keys_counter}
good key(s)" if (bad_keys_counter + good_keys_counter) % 1000 == 0
end



--
View this message in context: http://riak-users.197444.n3.nabble.com/Problem-with-deleting-keys-tp3069985p4031578.html
Sent from the Riak Users mailing list archive at Nabble.com.




More information about the riak-users mailing list