cloned machines issues
shane at mcewan.id.au
Mon Apr 29 06:57:28 EDT 2013
On 27/04/13 10:56, Pieter Breed wrote:
> I'm trying to get two cloned machines to form a new cluster but I'm
> struggling to get rid of the old cluster config.
I went through the same issue when trying to clone our production
cluster to use as a staging cluster.
The trick is to configure the new cluster completely independently of
the one you're cloning and only copy the data directories (for LevelDB,
anyway. I don't know if it works with other backends.). Don't copy any
of the configuration files from the original cluster.
Here is how I did it:
* Create a new, empty cluster on the staging servers.
* Shut down Riak on the new cluster.
* If possible, shut down Riak on the production cluster to ensure a
* Copy the Production "leveldb" data directory to the staging servers.
NOTE: To avoid unnecessary handoffs between staging nodes make sure you
clone the nodes in the same order as the 'ringready' command lists them.
First production node to first staging node, etc.
* Start Riak on the staging cluster.
* Riak will start using the leveldb data as if it was its own. There's
nothing in the leveldb data directories that ties the data to a
particular node name.
* Monitor the Riak logs for errors or handoffs. Excessive handoffs means
you've cloned one or more production nodes to the wrong staging nodes.
Riak will fix it but it's inefficient.
I now automatically update our staging nodes every night with an rsync
of our production snapshot backups. This has two advantages: First, it
gives us a test platform with real, almost current, production data; and
second, it tests our backup system ensuring that our snapshots actually
work when loaded into Riak.
More information about the riak-users