riak bitcask calculation

Travis Kirstine tkirstine at firstbasesolutions.com
Mon Jul 18 11:54:45 EDT 2016


Yes, the reason I'm concerned is that we projected much lower memory usage based on the calculations.   We originally provisioned 2x the required memory and it appears that this will not be enough.

Am I correct in that the top cmd's RES memory for the beam.smp command is the memory being used by riak for "key storage", if the server was to be rebooted the memory would eventually climb back to this level?

Thanks for your help 

-----Original Message-----
From: Luke Bakken [mailto:lbakken at basho.com] 
Sent: July-18-16 11:35 AM
To: Travis Kirstine <tkirstine at firstbasesolutions.com>
Cc: riak-users at lists.basho.com; achan at jdbarnes.com
Subject: Re: riak bitcask calculation

Hi Travis -

The calculation provided for bitcask memory consumption is only a rough guideline. Using more memory than the calculation suggests is normal and expected with Riak. As you increase load on this cluster memory use may go up further as the operating system manges disk operations and buffers.

Is there a reason you're concerned about this usage?

--
Luke Bakken
Engineer
lbakken at basho.com


On Mon, Jul 18, 2016 at 8:28 AM, Travis Kirstine <tkirstine at firstbasesolutions.com> wrote:
> Yes from the free command
>
> [root at riak1 ~]# free -g
>               total        used        free      shared  buff/cache   available
> Mem:             45           9           0           0          36          35
> Swap:            23           0          22
>
> Or from top
>
> PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
> 24421 riak      20   0 16.391g 8.492g  41956 S  82.7 18.5  10542:10 beam.smp
>
>
> I don't think that we are IO bound
> dstat
>
> ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
> usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
>   0   0  99   0   0   0| 150k  633k|   0     0 |   1B   25B| 702  2030
>   0   5  95   0   0   0|   0     0 |  10k 2172B|   0     0 |1125   765
>   2   6  92   0   0   0|   0     0 | 213k  135k|   0     0 |2817  7502
>   2   5  92   0   0   0|   0     0 | 159k   88k|   0     0 |2758  9834
>   2   5  93   0   0   0|   0  4884k| 278k   70k|   0     0 |2923  7453
>   0   5  95   0   0   0|4096B   10M|  21k 1066B|   0     0 |3121   781
>   4   7  89   0   0   0|   0    10M| 258k  160k|   0     0 |  13k   16k
>   0   5  95   0   0   0|   0  4096B| 200k   65k|   0     0 |1413  1589
>   1   5  92   1   0   0|   0    26k| 287k  206k|   0     0 |2124  4990
>   1   4  95   0   0   0|   0  2048B|  67k   78k|   0     0 |1667  4504
>   1   4  95   0   0   0|   0  1560k| 102k  105k|   0     0 |1639  4146
>   3   8  88   1   0   0|   0    86M| 453k  335k|   0     0 |6097    16k
>   4  14  81   0   0   0|   0    15k| 635k  564k|   0     0 |5383    14k
>   0   4  96   0   0   0|   0     0 |  29k 1697B|   0     0 |1121   769
>   4   7  89   0   0   0|   0     0 | 339k  376k|   0     0 |8017    15k
>   5  16  79   0   0   0|   0    11M| 847k  824k|   0     0 |  13k   30k
>   2  12  86   1   0   0|4096B   10M| 301k  272k|   0     0 |4639    11k
>   3  10  87   0   0   0|   0    10M| 508k  610k|   0     0 |8260    17k
>   2   9  87   2   0   0|   0    13k| 523k  354k|   0     0 |3432    10k
>   0   4  96   0   0   0|   0     0 |3434B 1468B|   0     0 |1063   774


More information about the riak-users mailing list