scp data

Daniel Reverri dan at basho.com
Fri Jul 29 16:18:25 EDT 2011


Hi Francisco 

When tarring up the data directories on nodes it's important to remember that the partition to node mapping as defined in the ring file will determine where a cluster looks for data.

Here is a method for migrating data to a new cluster:

Using "riak-admin reip" to restore Bitcask tarballs to a new cluster

Backup process:
1. Tarball the Bitcask and ring data directories for each node
* Package installed versions of Riak usually place this data in "/var/lib/riak/bitcask" and "/var/lib/riak/ring"


Restore process:
1. Deploy a set of Riak nodes; same number of nodes as the backed up cluster
* Don't start or cluster the nodes

2. Restore the Bitcask and ring data to the new set of nodes

3. On each node run "riak-admin reip" for all nodes (explained in the example below)

4. Start the Riak nodes
* there is no need to cluster the nodes; the restored ring file will form the cluster


Example:
	We have a cluster with 4 nodes (1, 2, 3, 4) which we want to backup and restore to 4 new nodes (A, B, C, D)

	Steps:
		Backup the Bitcask and ring data directories for nodes 1, 2, 3, 4
		Install Riak on nodes A, B, C, D
		On node A:
			Restore the Bitcask and ring data directories from node 1
			Reip the ring file:
				riak-admin reip node1 nodeA
				riak-admin reip node2 nodeB
				riak-admin reip node3 nodeC
				riak-admin reip node4 nodeD
		Repeat on nodes B, C, D using the back ups from 2, 3, 4 respectively
			* The reip commands will be the same on all nodes
		Start nodes A, B, C, D
		Nodes A, B, C, D will form a cluster that is a clone of nodes 1, 2, 3, 4


Limitations:
	Backups must be restored to the same number of nodes from which they were taken

Thanks
Dan

Sent from my iPhone

On Jul 29, 2011, at 3:22 PM, francisco treacy <francisco.treacy at gmail.com> wrote:

> Hi all,
> 
> I'm setting up a brand-new staging server, with Riak 0.14.2.
> 
> As I want to get all production data, I thought I could scp the data
> directory into the new one (besides the ring dir).
> 
> So that's what I did, and then started the staging db for the first
> time. It doesn't really work though. Many documents are present, but
> for others I get 404s and possibly 500s as I see tons of crashes in
> the logs (certainly many related to Luwak).
> 
> I believed this kind of operation was legal... Is there something I'm
> missing, or something that could help?
> 
> Ideally, I want to transfer fresh production data to staging via a
> cron job on a daily basis.
> 
> Thanks,
> Francisco
> 
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110729/896f64cf/attachment.html>


More information about the riak-users mailing list