Riak behind a Load Balancer

Michael Clemmons glassresistor at gmail.com
Mon Jun 25 15:01:11 EDT 2012


If your running multiple clients use an LB on each and do failover if the
local lb is down.

On Mon, Jun 25, 2012 at 11:41 AM, Swinney, Austin <Austin at vimeo.com> wrote:

>  I use the Amazon Elastic Load Balancer (ELB) on my ec2 riak cluster.  I
> understand the concerns of LB fail, but for me, using Riak is largely about
> ease of use.  Easy to deploy, grow, shrink, replace, etc.  Using an the ELB
> allows me to do those things without consulting those who manage the
> services that use cluster.  And in the turbulent cloud space, if a node
> goes unresponsive, it just gets kicked out without any action on my part.
>
>  A few weeks ago, I had some episode where I replaced all the nodes in
> the riak cluster.  It sucked for me (because I don't know what I'm doing!),
> but the service owner who actually uses the cluster came to my desk a week
> later and asked, "So how is everything going with the riak cluster?"  He
> hadn't even noticed, nor had anyone else unless I told them about it.  To
> me, that is pure ops gold.
>
>  Performance wise, I get what I need out of it.  Response times through
> the ELB are all pretty similar across the board.  I never tested it without
> the ELB, so I am not aware of any performance hit using ELB.
>
>  My ELB health check settings:
>
>      Ping Target:
> HTTP:8098/ping
>    Timeout:
> 5 seconds
>    Interval:
> 30 seconds
>    Unhealthy Threshold:
> 2
>    Healthy Threshold:
> 10
>
>     If I didn't have ELB, I'd look at the zookeeper route.
>
>  On Jun 25, 2012, at 2:21 PM, Anthony Molinaro wrote:
>
>  This is almost exactly what we do with our riak clusters (as well as
> actually
> all our subclusters).  It's actually nice for developer because when they
> need to talk to a network service they just reference 127.0.0.1 and some
> port, and haproxy gets them to the right place.  This also allows an
> operations team to make network changes without impacting applications
> (via the use of the haproxy reload).
>
> The only downside is that if your connections are long lived and you
> restart
> your riak nodes, all requests will end up on the last node.  And if you
> restart
> haproxy they will all end up on the first node initially.  I sort of wish
> there was a configuration parameter to haproxy to randomize the start
> point for the roundrobin so you could keep things a little more balanced
> across many machines running haproxy.
>
> -Anthony
>
> On Mon, Jun 25, 2012 at 07:44:05AM -0400, Sean Cribbs wrote:
>
> Another typical setup is to have each client node have its own haproxy, and
>
> when Riak nodes are added or removed (not a common occurrence, mind you), a
>
> configuration management tool like Chef/Puppet/cfengine/etc can adjust the
>
> config and signal the process to reload it (I think it's `kill -HUP`). Then
>
> your client code also only ever needs to connect to localhost, and doesn't
>
> have to have itself reconfigured.
>
>
>  On Mon, Jun 25, 2012 at 4:40 AM, Samuel Elliott <sam at lenary.co.uk> wrote:
>
>
>  On Mon, Jun 25, 2012 at 7:36 AM, Matt Black <matt.black at jbadigital.com>
>
>  wrote:
>
>   Dear list,
>
>
>    Does anyone have an opinion on the concept of putting a Riak cluster
>
>   behind
>
>   a load balancer?
>
>
>   It has been done before. there are various results when searching
>
>  "riak haproxy" in your favourite search engine.
>
>
>
>    We wish to be able to automatically add/remove nodes from the cluster,
> so
>
>   adding an extra layer at the front is desirable. We should also benefit
>
>   for
>
>   incoming requests behind shared across all nodes.
>
>
>    Can anyone see any drawbacks / problems with doing this?
>
>
>   If your load balancer falls over, what do you do then? Highly
>
>  available may go down the pan. Have more than one would be the obvious
>
>  answer.
>
>
>   What do you do when you want to transparently add more machines to
>
>  your load balancer?
>
>
>   Maybe it might be better to have a list of riak nodes stored in a
>
>  separate registry (I'm thinking something like zookeeper), that your
>
>  application servers can then poll for changes (or even subscribe to
>
>  changes) to the list of servers.
>
>
>   Sam
>
>
>
>    Thanks
>
>   Matt
>
>
>
>    _______________________________________________
>
>   riak-users mailing list
>
>   riak-users at lists.basho.com
>
>   http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>
>
>
>   --
>
>  Samuel Elliott
>
>  sam at lenary.co.uk
>
>  http://lenary.co.uk/
>
>  +44 (0)7891 993 664
>
>
>   _______________________________________________
>
>  riak-users mailing list
>
>  riak-users at lists.basho.com
>
>  http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>
>
>
>  --
>
> Sean Cribbs <sean at basho.com>
>
> Software Engineer
>
> Basho Technologies, Inc.
>
> http://basho.com/
>
>
> _______________________________________________
>
> riak-users mailing list
>
> riak-users at lists.basho.com
>
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>
> --
> ------------------------------------------------------------------------
> Anthony Molinaro                           <anthonym at alumni.caltech.edu>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>


-- 
-Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20120625/f7c8237b/attachment.html>


More information about the riak-users mailing list