Beginners performance problem

Dmitri Zagidulin dzagidulin at basho.com
Fri Feb 22 17:20:35 EST 2013


Lars,
If increasing number of worker threads and using connection pooling
did not improve performance, then maybe we're looking at some kind of
environmental or setting issue.

But just in case -- can you post snippets of the code that's setting
up the java riak client and issuing the writes?


On Fri, Feb 22, 2013 at 5:09 PM, Lars J. Nilsson
<lars.j.nilsson at cubeia.com> wrote:
> Sorry, yes. I've used anything from single threads to one thread per
> "actor". Including anything from one connection to one per process, and
> pooling, etc.
>
> /Lars J. Nilsson
> [sent from my mobile]
>
> Den 22 feb 2013 19:49 skrev "Sean Cribbs" <sean at basho.com>:
>
>> Use multiple threads in your Java program to parallelize the writes.
>>
>> On Fri, Feb 22, 2013 at 9:40 AM, Lars J. Nilsson
>> <lars.j.nilsson at cubeia.com> wrote:
>> > Hi Sean,
>> >
>> > Thanks for pipjng in. And how do I do that?
>> >
>> > Cheers
>> > /Lars J. Nilsson
>> > Executive VP, Cubeia Ltd
>> > +46 (0) 704 / 10 69 53
>> >
>> >
>> > On 22 February 2013 16:28, Sean Cribbs <sean at basho.com> wrote:
>> >>
>> >> Given iostat says ~10% utilization, I would suggest increasing the
>> >> concurrency of your writes. It's obviously not using up the capacity
>> >> of the disk, your bottleneck is elsewhere. 2MB/sec is low for any
>> >> commodity hardware.
>> >>
>> >> On Fri, Feb 22, 2013 at 9:21 AM, Lars J. Nilsson
>> >> <lars.j.nilsson at cubeia.com> wrote:
>> >> > Hi all,
>> >> >
>> >> > I'm micro-testing a simple update pattern (key + short json document)
>> >> > on
>> >> > my
>> >> > local workstation (8 core intel CPU, 8 gig of RAM, 10K disk) to get
>> >> > baseline
>> >> > before moving to cluster on AWS. With Riak I get about 60 mods/s
>> >> > before
>> >> > maxing out (as compared to approx 3000 mods/s on a vanilla MySQL
>> >> > using
>> >> > exactly the same code).
>> >> >
>> >> > It feels like I must be doing something hideously wrong. (For the
>> >> > record,
>> >> > I'm a decent Java programmer but a riak/erlang n00b).
>> >> >
>> >> > Riak is installed via a vanilla Ununtu 12.04 package as described in
>> >> > the
>> >> > docs. The test ran on the same disk as the mentioned MySQL test. The
>> >> > same
>> >> > Java code was doing both tests (just a DAO swapped).
>> >> >
>> >> > The bucket was using n_val = 1.
>> >> >
>> >> > The problem seems to be in high disk-writing, despite the low load.
>> >> > Here's
>> >> > what iostat says during the run:
>> >> >
>> >> > [...]
>> >> > wrqm/s : 434.50
>> >> > w/s: 182.90
>> >> > wkB/s: 2618.00
>> >> > avgrq-sz: 28.63
>> >> > [...]
>> >> > %util: 10.60
>> >> >
>> >> > Which seems to be too much considering the small load (the stored
>> >> > JSON
>> >> > blobs
>> >> > are roughly 1K each).
>> >> >
>> >> > I'm using the official PB Java client.
>> >> >
>> >> > This is an evaluation for a customer. Any hint of what I'm missing or
>> >> > what
>> >> > the problem could be would be appreciated.
>> >> >
>> >> > Cheers
>> >> > /Lars J. Nilsson
>> >> > Executive VP, Cubeia Ltd
>> >> > +46 (0) 704 / 10 69 53
>> >> >
>> >> > _______________________________________________
>> >> > riak-users mailing list
>> >> > riak-users at lists.basho.com
>> >> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Sean Cribbs <sean at basho.com>
>> >> Software Engineer
>> >> Basho Technologies, Inc.
>> >> http://basho.com/
>> >
>> >
>>
>>
>>
>> --
>> Sean Cribbs <sean at basho.com>
>> Software Engineer
>> Basho Technologies, Inc.
>> http://basho.com/
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>




More information about the riak-users mailing list