Does RiakKV require a lot of memory?

Luke Bakken lbakken at basho.com
Mon Mar 27 11:25:43 EDT 2017


Hi Tsutomu,

You should only use objects up to a maximum of 1MiB in size with Riak
KV. If you wish to store larger objects or files, please use Riak CS.

--
Luke Bakken
Engineer
lbakken at basho.com

On Sun, Mar 26, 2017 at 5:38 PM,  <wakuda_tsutomu at intec.co.jp> wrote:
> Hello.
>
> The following problem occurred, Please give me on solution.
>
> (Problem)
> I registered a 10mb to 1000mb file.
> Then RiakKV hung up when the batch program was accessing files from 100 MB to 600 MB.
> RiakKV hung up due to insufficient memory.
>
> (Question)
> 1.The problem that occurred this time is the effect of data accessed to a 20-1000 MB file?
>   Or is it another problem?
>
> 2.During RiakKV data update, when you want to access RiakKV data,
>   do you need to do something special?
>   Do you need a lot of memory?
>
> (Objects)
> file: 3,882,892
> file size(total): 332.46GB(346,243,500,913byte)
> file objects:
>   1-  10MB = 3,800,000(all)
>  20-  50MB = 40 to 100
> 100-1000MB = 50
>
> (Riak Sever)
> OS:Red Hat Enterprise Linux Server release 6.7 (Santiago)
>  (Linux patdevsrv02 2.6.32-573.el6.x86_64)
> CPU:Intel(R) Xeon(R) CPU E5640  @2.67GHz * 2
> Memory:12GB
> Swap:14GB
> Network:1Gbps
> Disk:
>  Filesystem      Size  Used Avail Use% Mounted on
>  /dev/sda3       261G   67G  182G  27% /
>  tmpfs           5.9G  300K  5.9G   1% /dev/shm
>  /dev/sda1       477M   71M  381M  16% /boot
>  /dev/sdb1       275G  243G   19G  93% /USR1
>  /dev/sdc1       1.7T  1.5T   76G  96% /USR2
>  /dev/sdd1       1.1T  736G  309G  71% /USR3 <<< Store
>  /dev/sde1       1.1T  1.1T   18G  99% /USR4
>  /dev/sdf1       1.4T  1.1T  365G  74% /media/USB-HDD1
>
> (Riak)
> RiakKV 2.2.0 (riak-2.2.0-1.el6.x86_64.rpm)
>
> (Riak Node)
> 1 Node.
>
> !!!We plan to add two nodes to the cluster at a later date.!!!
>
> (Java)
> Java1.8 (jre-8u121-linux-x64.rpm)
>
> (riak setting)
> /etc/riak/riak.conf
> storage_backend = leveldb
> leveldb.maximum_memory.percent = 50
> object.size.maximum = 2GB
> listener.http.internal = 0.0.0.0:8098
> listener.http.internal = 0.0.0.0:8098
> platform_data_dir = /USR3/riak
> nodename = riak at ...
> riak_control = on
>
> !!!Settings other than these remain the default.!!!
>
> (linux setting)
> /etc/security/limits.conf
> * soft nofile 65536
> * hard nofile 65536
>
> Thank you.
>
> Tsutomu Wakuda
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com




More information about the riak-users mailing list