riak bitcask calculation

Luke Bakken lbakken at basho.com
Mon Jul 18 11:13:52 EDT 2016


Hi Travis,

Could you go into detail about how you're coming up with 9GiB per
node? Is this from the output of the "free" command?

Bitcask uses the operating system's buffers for file operations, and
will happily use as much free ram as it can get to speed up
operations. However, the OS will use that memory for other programs
that need it should that need arise.

If you're using Linux, these settings may improve bitcask performance
in your cluster:

http://docs.basho.com/riak/kv/2.1.4/using/performance/#optional-i-o-settings

Benchmarking before and after making changes is the recommended way to proceed.

Thanks -

--
Luke Bakken
Engineer
lbakken at basho.com


On Fri, Jul 15, 2016 at 8:48 AM, Travis Kirstine
<tkirstine at firstbasesolutions.com> wrote:
> I've put ~74 million objects in my riak cluster with a bucket size of 9
> bytes and keys size of 21 bytes.  According the riak capacity calculator
> this should require ~4 GiB of RAM per node.  Right now my servers are
> showing ~ 9 GiB used per node.  Is this caused by hashing or something
> else......
>
> # capacity calculator output
>
> To manage your estimated 73.9 million key/bucket pairs where bucket names
> are ~9 bytes, keys are ~21 bytes, values are ~36 bytes and you are setting
> aside 16.0 GiB of RAM per-node for in-memory data management within a
> cluster that is configured to maintain 3 replicas per key (N = 3) then Riak,
> using the Bitcask storage engine, will require at least:
>
> 5 nodes
> 3.9 GiB of RAM per node (19.7 GiB total across all nodes)
> 11.4 GiB of storage space per node (56.8 GiB total storage space used across
> all nodes)




More information about the riak-users mailing list