Using the 'fold' keys threshold

Alex Moore amoore at basho.com
Thu Jun 5 11:34:51 EDT 2014


Hi Venkat,

You can find those settings in our docs: http://docs.basho.com/riak/1.4.9/ops/advanced/backends/bitcask/#Configuring-Bitcask (search for “Fold Keys Threshold”).   

In Bitcask when we do range operations like “List Keys” or other operations that require us to fold over all the data, we take a snapshot of the “Keydir” to get a consistent read.  The Keydir is the hash table that holds the `key-> latest object` mapping.  When we do this snapshot, we also start a delta of any changes since the snapshot.

We use the two “Fold Keys Threshold” options `max_fold_age` and `max_fold_puts` only when Bitcask is processing one fold operation, and gets a request for a second one.  These two options let the user choose whether to reuse the snapshot, or to block and get a new snapshot before starting the second fold.  This lets you tradeoff between a potential performance boost and consistency.

By default we have Bitcask side toward consistency; it will get a new snapshot, by setting `max_fold_puts` to `0`.  If any new puts come in, we must grab a new snapshot before folding again. 

 	- Increasing `max_fold_puts` to `n` will let Bitcask reuse the snapshot if there are fewer than `n` changes in the delta.
 	- Increasing `max_fold_age` to `s` will let Bitcask reuse the snapshot if the snapshot is younger than `s` microseconds. 

Setting either of these to positive values can let folds ignore recent changes, so you can run into stale data. Because of that, we recommend that you don’t change them.
I hope this helps.

Thanks,
Alex


On May 13, 2014, at 3:02 PM, Venkatachalam Subramanian <venkatsubbu44 at gmail.com> wrote:

> Hi All,
> 
> It was very helpful to get my first few questions about Riak/Bitcacsk answered pretty quickly.
> 
> I just have a another question on the same lines,
> 
> I ran across the 'fold keys threshold' option in riak/bitcask.
> I could not find enough information about the 'fold keys' option to understand it completely.
> 
> Could someone tell me what 'fold keys' option is? what does it do? when could we use it? 
> Does it help when you want to get the list of all keys available?
> 
> I greatly appreciate your help.
> Thank You.
> 
> -- 
> Regards,
> Venkat Subramanian
> 
> 
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20140605/549c347b/attachment.html>


More information about the riak-users mailing list