Tune Riak for fast inserts - populate DB

Bogdan Flueras flueras.bogdan at gmail.com
Thu Feb 14 04:51:14 EST 2013


What I have is:
Network bandwidth between nodes: 18.4 MB/sec
Network cards: 1.000Mbps full duplex
Riak process: IO: avg 600 K/s for disk write, 7% RAM used, 17% CPU used

Doesn't seem notwork bottleneck.

ing. Bogdan Flueras



On Thu, Feb 14, 2013 at 11:21 AM, Bogdan Flueras
<flueras.bogdan at gmail.com>wrote:

> What I have is:
>
> Network bandwidth between nodes: 18.4 MB/sec
> Network cards: 1.000Mbps full duplex
> Riak process: IO: avg 600 K/s for disk write, 7% RAM used, 17% CPU used
>
>
> ing. Bogdan Flueras
>
>
>
> On Thu, Feb 14, 2013 at 10:17 AM, Erik Søe Sørensen <ess at trifork.com>wrote:
>
>> Just to rule something out: what is the capacity of the network between
>> the cluster and the client - and among the cluster nodes?
>>
>>
>> -------- Oprindelig meddelelse --------
>> Fra: Shuhao <shuhao at shuhaowu.com>
>> Dato:
>> Til: riak-users at lists.basho.com
>> Emne: Re: Tune Riak for fast inserts - populate DB
>>
>>
>> Hi,
>>
>> I've had similar experiences with riak. I used the Riak Python client
>> with HTTP and PBC (even rolled my own client! That only had a ~10% speed
>> up though, and it was HTTP only). I've also examined that the Riak host
>> machine's CPU is only at around 70% when I insert. Other NoSQL dbs uses
>> about 100% when I just continuous insert. Some DBs even have bulk insert.
>>
>> Maybe this is an issue with Riak itself... I don't really know. Maybe a
>> Riak dev could chip in on the conversation
>>
>> Cheers,
>>
>> Shuhao
>>
>> On 13-02-13 02:37 AM, Bogdan Flueras wrote:
>> > Hello all,
>> > I've got a 5 node cluster with Riak 1.2.1, all machines are multicore,
>> > with min 4GB RAM.
>> >
>> > I want to insert something like 50 million records in Riak with the java
>> > client (Protobuf used) with default settings.  I've tried also with HTTP
>> > protocol and set w = 1 but got some problems.
>> >
>> > However the process is very slow: it doesn't write more than 6GB/ hour
>> or
>> > aprox. 280 KB/second.
>> > To have all my data filled in, it would take aprox 2 days !!
>> >
>> > What can I do to have the data filled into Riak ASAP?
>> > How should I configure the cluster ? (vm.args/ app.config) I don't care
>> so
>> > much about consistency at this point.
>> >
>> > Thank you,
>> > ing. Bogdan Flueras
>> >
>> >
>> >
>> > _______________________________________________
>> > riak-users mailing list
>> > riak-users at lists.basho.com
>> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> >
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130214/5569508f/attachment.html>


More information about the riak-users mailing list