how to really delete keys from Riak

Alexander Sicular siculars at gmail.com
Wed Apr 20 15:32:40 EDT 2016


Check out bitcask TTL, expiry. 
-Alexander 


@siculars
http://siculars.posthaven.com

Sent from my iRotaryPhone

> On Apr 20, 2016, at 12:20, Shifeng Xiao <xiaoshifeng84 at gmail.com> wrote:
> 
> Thanks for the information Alexander and Daniel, I appreciated it.
> 
> We are using Riak2.0 and bitcask as backend, and we are writing the keys once into bucket, keep it for a few hours, then delete it, never update the existing data.
> 
> 
>> On Wed, Apr 20, 2016 at 11:32 AM, Alexander Sicular <siculars at gmail.com> wrote:
>> Hello Shifeng,
>> 
>> Riak uses Bitcask as its default data backend, it may also use leveldb
>> as a backend but must be configured to do so[0][1]. Both bitcask[2]
>> and leveldb[3] do not reclaim disk space immediately when a delete is
>> issued. Incidentally, neither do updates. Because Riak is
>> non-destructive, aka append only, aka no in-place updates, mutation
>> operations like updates or deletes are actually writes. For example, a
>> 10KB object that is updated four times in some short interval will
>> consume 50KB space (original 10KB + 4 x 10KB updates) until
>> merge/compaction occurs. A delete is actually a special zero byte
>> value write indicating a key is marked for deletion, aka a tombstone.
>> Bitcask will reclaim disk space via merge operations which may be
>> controlled by the user. Leveldb reclaims disk space via compaction
>> operations.
>> 
>> Architects and Developers should take into account their update/delete
>> ratios and consider how those will affect disk space consumption and
>> consider how deletes operate in distributed systems in general and
>> Riak in specific from an application perspective. John Daily wrote a
>> must read four part series on Riak's configurable behaviors that all
>> Riak developers should be familiar with. Part 3 deals specifically
>> with delete related considerations[4].
>> 
>> Thank you,
>> Alexander
>> 
>> [0] http://docs.basho.com/riak/kv/latest/setup/planning/backend/leveldb/#installing-eleveldb
>> [1] http://docs.basho.com/riak/kv/latest/configuring/backend/
>> [2] http://docs.basho.com/riak/kv/latest/setup/planning/backend/bitcask/
>> [3] http://docs.basho.com/riak/kv/latest/setup/planning/backend/leveldb/
>> [4] http://basho.com/posts/technical/riaks-config-behaviors-part-3/
>> 
>> On Wed, Apr 20, 2016 at 8:17 AM, Daniel Abrahamsson
>> <daniel.abrahamsson at klarna.com> wrote:
>> > Hi,
>> >
>> > What riak version and what riak backend are you using? For leveldb you have
>> > to take compaction into account, and for older bitcask versions there are
>> > known issues with data not being properly deleted.
>> >
>> > //Daniel
>> >
>> > _______________________________________________
>> > riak-users mailing list
>> > riak-users at lists.basho.com
>> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> >
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20160420/0f0fdeca/attachment-0002.html>


More information about the riak-users mailing list