standby cluster experiment

John Loehrer jloehrer at gaiaonline.com
Tue Dec 20 00:10:03 EST 2011


Yeah, I'll probably end up doing something like that. I even thought of having haproxy monitor special keys in riak:

 http://riakcluster:8098/riak/serverstatus/<nodename>

I can then have each node delete its own key during snapshot mode and add it back when done.  The only tricky thing about it is creating a clean way to cycle through and back up each node in turn. That is part of what I liked so much about the idea of a hot backup with lvm ... no need to take the node out of rotation. The lvm snapshot creation is near instant and cheap, and then the rsync backup I was using could be niced to be very low priority but very quick. If only I could somehow tell leveldb to quickly pause or close out any open log files or somehow get it to repair on startup in the standby node.

In any case, I have a workable if not ideal solution. Thanks for the help.

~ John

----- Quoted Response -----

> Haproxy has a standby system you can use to remove a node from rotation 
> politely, allowing existing requests to finish. You can remove them from 
> haproxy directly at the command line (or using http-check 
> disable-on-404, but that doesn't really make sense for riak)




More information about the riak-users mailing list