Import/Export and Bulk operations

Kevin Burton rkevinburton at charter.net
Tue Feb 26 18:57:35 EST 2013


This worked just fine on a Joyent server but when I try it on an AWS hosted
server I get:

 

Port not specified, using the default: 8087

HTTP port not specified, using the default: 8098

Could not connect to Riak on PB port 8087

 

Do you have any experience with using this tool on an AWS machine? I looked
at the AWS security group and it looks like port 8087 is open. I am at  a
loss.

 

From: riak-users [mailto:riak-users-bounces at lists.basho.com] On Behalf Of
Dmitri Zagidulin
Sent: Tuesday, February 26, 2013 2:53 PM
To: riak-users
Subject: Re: Import/Export and Bulk operations

 

Hi Kevin,

Good questions, all.

1) The warning about app.config values being the same only applies to things
like "if you're using secondary indices on one cluster, you better also turn
them on on the second cluster, and also be running leveldb, etc" or "if
you're using custom quorum values in your config, you better also have the
same values in the other cluster you're migrating to". They don't literally
need to be the same app.config files -- maybe I can clarify that in the
docs. So basically, it's fine, as long as you use common sense, and turn on
the same things in the other cluster that you need for your app code.

2) If you're running the export on one of the nodes of your source cluster,
I think you might have better luck using localhost instead of the
public-facing hostname. So, try something like:
java -jar riak-data-migrator-0.1.3.jar  -d -a -r ~/riak-dump -h
riak at 127.0.0.1 <mailto:riak at 172.16.33.107> 

I don't know your networking environment or firewall rules, but it may
easily be the case that port 8087 might not be open from the outside (via
the public IP), even from inside the node. Anyhow, try the localhost route,
and if that doesn't work, do make sure that the port is actually open in the
firewall (and accessible locally).

Hope that helps.

Dmitri

On Tue, Feb 26, 2013 at 3:04 PM, Kevin Burton <rkevinburton at charter.net>
wrote:

I have a question about the usage of this tool. I have data locally that I
would like to export to another riak server to test the dataset. The local
"server" is a cluster and the machine I want to transfer to is just a single
machine. I notice in the documentation that there is a "warning" about the
app.config values must be the same on the two machines before this tool can
be used. Naturally since I have a cluster of 4 machines as the source and
only one machine as the destination the app.config will be different. Is
this a problem.  

The cluster of machines all have basically private addresses the destination
server has a public IP address. So I want to transfer from the source to the
destination. I am assuming that I would run the tool on one of the source
clusters and the -h option (for the host) would be the public IP address of
the single Riak server. Right?

When I run it from the source to create file dumps with:

 

java -jar riak-data-migrator-0.1.3.jar  -d -a -r ~/riak-dump -h
riak at 172.16.33.107

 

I get the following output:

 

Port not specified, using the default: 8087

HTTP port not specified, using the default: 8098

Could not connect to Riak on PB port 8087

 

So apparently it throw an error. If I look in the app.config it seems that
the port is enabled. Any idea what the problem is?

 

{riak_api, [

. . . .

            {pb_port, 8087 }

            ]},

 

From: riak-users [mailto:riak-users-bounces at lists.basho.com] On Behalf Of
Dmitri Zagidulin
Sent: Friday, February 22, 2013 10:02 AM
To: riak-users
Subject: Re: Import/Export and Bulk operations

 

Ok, the 0.1.4 binary download link should be working now.

On Fri, Feb 22, 2013 at 9:13 AM, Chris Read <chris.read at gmail.com> wrote:

The README on the git repo refers to version 0.1.4, but the link to the
download is broken...

 

On Wed, Feb 20, 2013 at 12:07 PM, Hector Castro <hector at basho.com> wrote:

Hi Kevin,

 

The riak-data-migrator [0] is a tool used by our own professional services
team for bulk data import/export. The prebuilt JAR can be downloaded here
[1].

 

[0] https://github.com/dankerrigan/riak-data-migrator

[1] http://ps-tools.data.riakcs.net:8080/riak-data-migrator-0.1.3-bin.tar.gz

 

--

Hector

 

On Wed, Feb 20, 2013 at 5:33 AM, Kevin Burton <rkevinburton at charter.net>
wrote:

 

I was wondering if there were any import/export or bulk data operations that
I was missing? I would like to "clone" a bucket from one machine to another
but I would like to avoid having to do it one item at a time.

 

_______________________________________________
riak-users mailing list
riak-users at lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

 


_______________________________________________
riak-users mailing list
riak-users at lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

 


_______________________________________________
riak-users mailing list
riak-users at lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130226/05b09c0d/attachment.html>


More information about the riak-users mailing list