Using the 'fold' keys threshold
amoore at basho.com
Thu Jun 5 11:34:51 EDT 2014
You can find those settings in our docs: http://docs.basho.com/riak/1.4.9/ops/advanced/backends/bitcask/#Configuring-Bitcask (search for “Fold Keys Threshold”).
In Bitcask when we do range operations like “List Keys” or other operations that require us to fold over all the data, we take a snapshot of the “Keydir” to get a consistent read. The Keydir is the hash table that holds the `key-> latest object` mapping. When we do this snapshot, we also start a delta of any changes since the snapshot.
We use the two “Fold Keys Threshold” options `max_fold_age` and `max_fold_puts` only when Bitcask is processing one fold operation, and gets a request for a second one. These two options let the user choose whether to reuse the snapshot, or to block and get a new snapshot before starting the second fold. This lets you tradeoff between a potential performance boost and consistency.
By default we have Bitcask side toward consistency; it will get a new snapshot, by setting `max_fold_puts` to `0`. If any new puts come in, we must grab a new snapshot before folding again.
- Increasing `max_fold_puts` to `n` will let Bitcask reuse the snapshot if there are fewer than `n` changes in the delta.
- Increasing `max_fold_age` to `s` will let Bitcask reuse the snapshot if the snapshot is younger than `s` microseconds.
Setting either of these to positive values can let folds ignore recent changes, so you can run into stale data. Because of that, we recommend that you don’t change them.
I hope this helps.
On May 13, 2014, at 3:02 PM, Venkatachalam Subramanian <venkatsubbu44 at gmail.com> wrote:
> Hi All,
> It was very helpful to get my first few questions about Riak/Bitcacsk answered pretty quickly.
> I just have a another question on the same lines,
> I ran across the 'fold keys threshold' option in riak/bitcask.
> I could not find enough information about the 'fold keys' option to understand it completely.
> Could someone tell me what 'fold keys' option is? what does it do? when could we use it?
> Does it help when you want to get the list of all keys available?
> I greatly appreciate your help.
> Thank You.
> Venkat Subramanian
> riak-users mailing list
> riak-users at lists.basho.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the riak-users