Tune Riak for fast inserts - populate DB

Shuhao shuhao at shuhaowu.com
Wed Feb 13 17:09:26 EST 2013


Hi,

I've had similar experiences with riak. I used the Riak Python client 
with HTTP and PBC (even rolled my own client! That only had a ~10% speed 
up though, and it was HTTP only). I've also examined that the Riak host 
machine's CPU is only at around 70% when I insert. Other NoSQL dbs uses 
about 100% when I just continuous insert. Some DBs even have bulk insert.

Maybe this is an issue with Riak itself... I don't really know. Maybe a 
Riak dev could chip in on the conversation

Cheers,

Shuhao

On 13-02-13 02:37 AM, Bogdan Flueras wrote:
> Hello all,
> I've got a 5 node cluster with Riak 1.2.1, all machines are multicore,
> with min 4GB RAM.
>
> I want to insert something like 50 million records in Riak with the java
> client (Protobuf used) with default settings.  I've tried also with HTTP
> protocol and set w = 1 but got some problems.
>
> However the process is very slow: it doesn't write more than 6GB/ hour or
> aprox. 280 KB/second.
> To have all my data filled in, it would take aprox 2 days !!
>
> What can I do to have the data filled into Riak ASAP?
> How should I configure the cluster ? (vm.args/ app.config) I don't care so
> much about consistency at this point.
>
> Thank you,
> ing. Bogdan Flueras
>
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>




More information about the riak-users mailing list