5-node low-cost, low-power Riak KV cluster -- this sound feasible?

Magnus Kessler mkessler at basho.com
Thu Apr 27 12:09:48 EDT 2017


On 27 April 2017 at 04:01, Lloyd R. Prentice <lloyd at writersglen.com> wrote:

> Hello,
>
> I'd like to build a five-node low-cost, low-power Riak KV cluster. Use
> case: learning Riak,  development, testing, storage of personal data.
>
> -- Based on pure hand-waving, I've plugged numbers that seem more than
> adequate for my purposes into the Riak Cluster Capacity Planning
> Calculator. Here's the recommendation:
>
> To manage your estimated 100.0 thousand key/bucket pairs where bucket
> names are ~10 bytes, keys are ~36 bytes, values are ~97.7 KiB and you are
> setting aside 2.0 GiB of RAM per-node for in-memory data management within
> a cluster that is configured to maintain 3 replicas per key (N = 3) then
> Riak, using the Bitcask storage engine, will require at least:
>
> 5 nodes
> 6.4 MiB of RAM per node (31.9 MiB total across all nodes)
> 5.6 GiB of storage space per node (28.0 GiB total storage space used
> across all nodes)
> Based on this, I'm considering the following hardware:
>
> 5 ODROID XU-4
> http://www.hardkernel.com/main/products/prdt_info.php?g_code=G143452239825
>
> Each with 8 Gb eMMC storage
>
> This provides a 64-bit 2 GHz processor and 2 Gb RAM per day node running
> Ubuntu 16.04 at total cost of something under total $65000 for the cluster.
>
> Does this sound like a feasible way to go? Any downsides?
>
> Many thanks,
>
> LRP
>
>
Hi Lloyd,

The hardware you've linked to is based on the ARM processor architecture.
Please be aware that Basho does not supply prebuilt binaries for this
architecture, and you would have to compile your own binaries.

If you simply want to learn how to operate a multi-node cluster you can
build a so-called DEVREL release from source. This will install a
configurable number of separately configured nodes on a standard OSX or
Linux work station and should be sufficient to experiment with most aspects
of operating a multi-node cluster.

As far as memory requirements for all but a small test work-load is
concerned, we recommend to provide at least 350 MB of memory per VNode if
using the leveldb backend in production. Typically, a Riak node should have
between 6 and 25 VNodes (depending on ring size), and a 5 node cluster in
its default configuration will have 13 or 12 VNodes per node. If you plan
to use Solr based search, add at least another 1 GB for Solr. I have
successfully run a test cluster with heavy Solr use on 8GB nodes.

That said, I am aware that with some extra work Riak can be built on ARM
hardware, such as RaspberryPI, and has been used for demonstration purposes
on this platform. You can certainly run a Riak cluster on constrained
hardware if needed for testing purposes. However, if you'd like to evaluate
performance characteristics, you should use a setup that mirrors an
eventual production setup more closely.

I hope this helps somewhat towards finding the best platform for you.

Kind Regards,

Magnus




-- 
Magnus Kessler
Client Services Engineer
Basho Technologies Limited

Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20170427/b4aa2367/attachment-0002.html>


More information about the riak-users mailing list