Parameter Planning (eleveldb)

Simon Effenberg seffenberg at team.mobile.de
Sun Feb 3 17:44:14 EST 2013


Hi Matthew,

thanks a lot!

So now I have:

6 nodes each having 32GB RAM:

vnode_working_memory = 16GB / 256 / 6 (50% of RAM devided by ringsize
devided by nodes) = 390 MB

open_file_memory =
   (max_open_files-10) * (
    184 + (104MB/2048) *
    (8 + ((16+14336)/2048 +1) *
    0.6
   )

Now I'm missing the max_open_files .. how to calculate it?
I'm missing also average_write_buffer_size (see my question in Step 4).

If I would use the default values for average_write_buffer_size the
max_open_files could be calculated like:

memory/vnode = average_write_buffer_size + cache_size +
open_file_memory + 20 MB
<=> (memory/vnode) - 20 MB - cache_size - average_write_buffer_size =
    open_file_memory

so with the default values:

open_file_memory = 390MB - 20MB - 8MB - 45MB = 317MB

and now max_open_files would be

open_file_memory = (max_open_files-10) * (184 + (104MB/2048) * (8 + ((16
                   +14336)/2048 +1) * 0.6 )
<=> (max_open_files-10) = open_file_memory / (184 + (104MB/2048) * (8 +
                          ((16+14336)/2048 +1) * 0.6 )
<=> max_open_files = open_file_memory / (184 + (104MB/2048) * (8 +
                     ((16+14336)/2048 +1) * 0.6 ) + 10
<=> max_open_files = 317MB / (184+53248*(8+67.8)) + 10
<=> max_open_files = 317MB / 4036382.4 + 10
<=> max_open_files ~= 92

That would be the maximum amount of open files a server can handle
(per vnode), am I right? But now, is this enough? Or how to calculate
50% temporary server loss (3 of 6) and how is the count of keys/values
is taking into account? I'm somehow lost :(

Cheers
Simon

On Sun, 3 Feb 2013 16:12:25 -0500
Matthew Von-Maszewski <matthewv at basho.com> wrote:

> First:  Step 2 is talking about how many vnodes exist on a physical server.  If your ring size is 256, but you have 8 servers … then your vnode count for step 2 is 32.
> 
> Second:  the 2048 is a constant forced by Google's leveldb implementation.  It is the portion of a file covered by a single bloom filter.  This calculation constant disappears with the upcoming 1.3 release.
> 
> Third:  yes there is a "block_size" parameter that is 4096.  Increase that only if you want to REDUCE the performance of the leveldb instance.  4096 is a very happy value.  We have customers and tests with 130K data values, all using 4096 block size.  The block_size only governs the minimum written (aggregate size of small values that must be written as one unit at minimum).
> 
> Use 104Mbyte for your average sst file size.  It is "good enough"
> 
> 
> I am not following the question stream for Step 4 and beyond.  Please state again.
> 
> Matthew
> 
> 
> 
> 
> On Feb 3, 2013, at 3:44 PM, Simon Effenberg <seffenberg at team.mobile.de> wrote:
> 
> > Hi,
> > 
> > I'm not sure if I understand this all well to calculate the memory
> > usage per file and other stuff.
> > 
> > The webpage tells me some steps but I'm completly unsure if I understand all parameters.
> > 
> > "Step 1: Calculate Available Working Memory"
> > 
> > taking the example:
> > 
> > leveldb_working_memory = 32G * (1 - .50) = 16G
> > 
> > "Step 2: Calculate Working Memory per vnode"
> > 
> > vnode_working_memory = leveldb_working_memory / vnode_count
> > 
> > vnode_count = 256
> > 
> > => vnode_working_memory = 16G / 256 = 64MB/vnode
> > 
> > also easy
> > 
> > "Step 3: Estimate Memory Used by Open Files"
> > 
> > open_file_memory =
> >   (max_open_files-10) * (
> >     184 + (average_sst_filesize/2048) *
> >     (8 + ((average_key_size+average_value_size)/2048 +1) *
> >     0.6
> >   )
> > 
> > so how do I know the average_sst_filesize (and what is this value exactly)
> > (and is 2048 for both /2048 true or 4096 in riak 1.2?) and how do I know
> > the max_open_files?
> > 
> > 
> > average_key_size could be 16byte (I have to ask someone but taking it for now)
> > average_value_size will be 14kbyte 
> > 
> > so for now
> > 
> > open_file_memory =
> >   (max_open_files-10) * (
> >     184 + (average_sst_filesize/2048) *
> >     (8 + ((16+14336)/2048 +1) *
> >     0.6
> >   )
> > 
> > (side question: should I increase the block_size because of the big average value size?
> > and also should I leave the cache_size at the default value like it was recommended?)
> > 
> > "Step 4: Calculate Average Write Buffer"
> > 
> > should I increase these values or not? If only two are held in memory and I have, as an
> > example, 32GB or RAM like in this scenario, shouldn't I increase it to something else?
> > 
> > "Step 5: Calculate vnode Memory Used"
> > 
> > memory/vnode = average_write_buffer_size + cache_size + open_file_memory + 20 MB
> > 
> > So for now I miss almost all 3 values :(.
> > 
> > To get an Idea:
> > 
> > - 3 buckets
> > - overall ~ 343347732 keys (but only 2/3 have 14kbyte in average)
> > 
> > 
> > Thx for help!
> > Simon
> > 
> > _______________________________________________
> > riak-users mailing list
> > riak-users at lists.basho.com
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> 


-- 
Simon Effenberg | Site Ops Engineer | mobile.international GmbH
Fon:     + 49-(0)30-8109 - 7173
Fax:     + 49-(0)30-8109 - 7131

Mail:     seffenberg at team.mobile.de
Web:    www.mobile.de

Marktplatz 1 | 14532 Europarc Dreilinden | Germany


Geschäftsführer: Malte Krüger
HRB Nr.: 18517 P, Amtsgericht Potsdam
Sitz der Gesellschaft: Kleinmachnow 




More information about the riak-users mailing list