Scaling Riak CS to hundreds or thousands of servers
jmartin at basho.com
Thu Jul 25 11:05:07 EDT 2013
Just one caveat on stanchion clustering -- I would say it is technically
possible to do so with some of the linux clustering tools, but we have no
official guides or support for such a configuration.
On Thu, Jul 25, 2013 at 10:56 AM, John Daily <jdaily at basho.com> wrote:
> Good questions, Andre, thanks for reaching out.
> > How is the Scaling of Raik CS when setting up a few hundred servers in a
> The upper limit of a cluster size depends on traffic load, hardware, and
> network capacity, but anything approaching 100 servers is likely to run
> into trouble due to inherent limitations in both Riak and the Erlang VM
> itself. An Erlang cluster is a full mesh, so the cluster overhead grows
> significantly with the number of servers.
> > Does it make sense, to build a cluster of that size, or is it
> recommended to have smaller pools of clusters and shard the Files over more
> than one cluster?
> Definitely should look at deploying multiple clusters.
> > What about stanchion as the potetial single point of failure, how to
> prevent it from corupting the System? If I got it right, Stanchion handles
> the IDs of Buckets and some user stuff. So if no further Buckets or users
> need to be created, there is no need for stanchion at that time and
> up-/downloading can go on as usual?
> You're correct: user accounts and user buckets are the reason we need a
> consensus system, and thus the reason Stanchion exists. For existing
> users/buckets, as you indicate, files can be transferred without any
> involvement from Stanchion.
> It is possible to cluster Stanchion using traditional cluster tools.
> (Why Amazon chose to use a global namespace for buckets is a mystery
> beyond my ken.)
> > As I want to use riak cs for large files (50MB-15GB) is it possible to
> raise the chunk size up to these Filesizes, to prevent the system from
> heavy network traffic?
> Definitely not. Erlang's distribution protocol will behave badly with
> objects that large.
> -John Daily
> Technical Evangelist
> Basho Technologies
> riak-users mailing list
> riak-users at lists.basho.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the riak-users