Beginners performance problem
sean at basho.com
Fri Feb 22 12:00:55 EST 2013
Use multiple threads in your Java program to parallelize the writes.
On Fri, Feb 22, 2013 at 9:40 AM, Lars J. Nilsson
<lars.j.nilsson at cubeia.com> wrote:
> Hi Sean,
> Thanks for pipjng in. And how do I do that?
> /Lars J. Nilsson
> Executive VP, Cubeia Ltd
> +46 (0) 704 / 10 69 53
> On 22 February 2013 16:28, Sean Cribbs <sean at basho.com> wrote:
>> Given iostat says ~10% utilization, I would suggest increasing the
>> concurrency of your writes. It's obviously not using up the capacity
>> of the disk, your bottleneck is elsewhere. 2MB/sec is low for any
>> commodity hardware.
>> On Fri, Feb 22, 2013 at 9:21 AM, Lars J. Nilsson
>> <lars.j.nilsson at cubeia.com> wrote:
>> > Hi all,
>> > I'm micro-testing a simple update pattern (key + short json document) on
>> > my
>> > local workstation (8 core intel CPU, 8 gig of RAM, 10K disk) to get
>> > baseline
>> > before moving to cluster on AWS. With Riak I get about 60 mods/s before
>> > maxing out (as compared to approx 3000 mods/s on a vanilla MySQL using
>> > exactly the same code).
>> > It feels like I must be doing something hideously wrong. (For the
>> > record,
>> > I'm a decent Java programmer but a riak/erlang n00b).
>> > Riak is installed via a vanilla Ununtu 12.04 package as described in the
>> > docs. The test ran on the same disk as the mentioned MySQL test. The
>> > same
>> > Java code was doing both tests (just a DAO swapped).
>> > The bucket was using n_val = 1.
>> > The problem seems to be in high disk-writing, despite the low load.
>> > Here's
>> > what iostat says during the run:
>> > [...]
>> > wrqm/s : 434.50
>> > w/s: 182.90
>> > wkB/s: 2618.00
>> > avgrq-sz: 28.63
>> > [...]
>> > %util: 10.60
>> > Which seems to be too much considering the small load (the stored JSON
>> > blobs
>> > are roughly 1K each).
>> > I'm using the official PB Java client.
>> > This is an evaluation for a customer. Any hint of what I'm missing or
>> > what
>> > the problem could be would be appreciated.
>> > Cheers
>> > /Lars J. Nilsson
>> > Executive VP, Cubeia Ltd
>> > +46 (0) 704 / 10 69 53
>> > _______________________________________________
>> > riak-users mailing list
>> > riak-users at lists.basho.com
>> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> Sean Cribbs <sean at basho.com>
>> Software Engineer
>> Basho Technologies, Inc.
Sean Cribbs <sean at basho.com>
Basho Technologies, Inc.
More information about the riak-users