Tune Riak for fast inserts - populate DB
Erik Søe Sørensen
ess at trifork.com
Thu Feb 14 03:17:02 EST 2013
Just to rule something out: what is the capacity of the network between the cluster and the client - and among the cluster nodes?
-------- Oprindelig meddelelse --------
Fra: Shuhao <shuhao at shuhaowu.com>
Til: riak-users at lists.basho.com
Emne: Re: Tune Riak for fast inserts - populate DB
I've had similar experiences with riak. I used the Riak Python client
with HTTP and PBC (even rolled my own client! That only had a ~10% speed
up though, and it was HTTP only). I've also examined that the Riak host
machine's CPU is only at around 70% when I insert. Other NoSQL dbs uses
about 100% when I just continuous insert. Some DBs even have bulk insert.
Maybe this is an issue with Riak itself... I don't really know. Maybe a
Riak dev could chip in on the conversation
On 13-02-13 02:37 AM, Bogdan Flueras wrote:
> Hello all,
> I've got a 5 node cluster with Riak 1.2.1, all machines are multicore,
> with min 4GB RAM.
> I want to insert something like 50 million records in Riak with the java
> client (Protobuf used) with default settings. I've tried also with HTTP
> protocol and set w = 1 but got some problems.
> However the process is very slow: it doesn't write more than 6GB/ hour or
> aprox. 280 KB/second.
> To have all my data filled in, it would take aprox 2 days !!
> What can I do to have the data filled into Riak ASAP?
> How should I configure the cluster ? (vm.args/ app.config) I don't care so
> much about consistency at this point.
> Thank you,
> ing. Bogdan Flueras
> riak-users mailing list
> riak-users at lists.basho.com
riak-users mailing list
riak-users at lists.basho.com
More information about the riak-users