Tune Riak for fast inserts - populate DB

Erik Søe Sørensen ess at trifork.com
Thu Feb 14 03:17:02 EST 2013


Just to rule something out: what is the capacity of the network between the cluster and the client - and among the cluster nodes?


-------- Oprindelig meddelelse --------
Fra: Shuhao <shuhao at shuhaowu.com>
Dato:
Til: riak-users at lists.basho.com
Emne: Re: Tune Riak for fast inserts - populate DB


Hi,

I've had similar experiences with riak. I used the Riak Python client
with HTTP and PBC (even rolled my own client! That only had a ~10% speed
up though, and it was HTTP only). I've also examined that the Riak host
machine's CPU is only at around 70% when I insert. Other NoSQL dbs uses
about 100% when I just continuous insert. Some DBs even have bulk insert.

Maybe this is an issue with Riak itself... I don't really know. Maybe a
Riak dev could chip in on the conversation

Cheers,

Shuhao

On 13-02-13 02:37 AM, Bogdan Flueras wrote:
> Hello all,
> I've got a 5 node cluster with Riak 1.2.1, all machines are multicore,
> with min 4GB RAM.
>
> I want to insert something like 50 million records in Riak with the java
> client (Protobuf used) with default settings.  I've tried also with HTTP
> protocol and set w = 1 but got some problems.
>
> However the process is very slow: it doesn't write more than 6GB/ hour or
> aprox. 280 KB/second.
> To have all my data filled in, it would take aprox 2 days !!
>
> What can I do to have the data filled into Riak ASAP?
> How should I configure the cluster ? (vm.args/ app.config) I don't care so
> much about consistency at this point.
>
> Thank you,
> ing. Bogdan Flueras
>
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>

_______________________________________________
riak-users mailing list
riak-users at lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


More information about the riak-users mailing list