Riak JAVA Client Performance

Pavel Kogan pavel.kogan at cortica.com
Wed Oct 10 11:19:27 EDT 2012


I understand that load balancing is a final solution, but I want to
benchmark single node.
If I knew that I can load single node with N requests / sec, I could assume
that after load balancing over 5 nodes my throughput limit will increase
linearly.

Pavel

On Wed, Oct 10, 2012 at 2:51 PM, Guido Medina <guido.medina at temetra.com>wrote:

>  The answer is there, create a client config with N pooled connections to
> your load balancer whatever you are using, I know HA proxy supports the PBC
> config (TCP based) which is faster than HTTP client, and hence my
> recommendation.
>
> Say, a non-clustered client config with N connections to balancer_host at
> 8087 and your balancer_host connected to EACH node, that's the way to go,
> the rest is about the CAP level you want to support which will impact your
> performance vs integrity. Up to you.
>
> CAP doc:
> http://docs.basho.com/riak/latest/tutorials/fast-track/Tunable-CAP-Controls-in-Riak/
>
> Guido.
>
>
> On 10/10/12 13:33, Pavel Kogan wrote:
>
> Hi,
>
>  The node is OK and not down.
> I have a way to do load balancing externally to JAVA Client.
> I am evaluating Riak for using in my company and want to measure maximal
> throughput vs single node.
>
>  Thanks,
>    Pavel
>
> On Wed, Oct 10, 2012 at 2:13 PM, Guido Medina <guido.medina at temetra.com>wrote:
>
>>  That question has been answered few times, here is my old answer:
>>
>> Hi,
>>
>>   It is the Java client which to be honest, doesn't handle well one node
>> going down, so, for example, in my company we use HA proxy for that, here is
>> a starting configuration: https://gist.github.com/1507077
>>
>>   Once we switched to HA proxy we just use a simple client without cluster
>> config, so the Java client doesn't know anything about the load balancing
>> going on. It works well, I can upgrade and restart servers without our Java
>> application be complaining.
>>
>> Regards,
>>
>> Guido.
>>
>>
>> On 10/10/12 12:58, Pavel Kogan wrote:
>>
>> Thanks,
>>
>>  I will try this solution.
>>
>>  Pavel
>>
>> On Wed, Oct 10, 2012 at 1:51 PM, kamiseq <kamiseq at gmail.com> wrote:
>>
>>> well I asked same question few days ago (maybe 2 weeks form now) and
>>> the answer was that yes sharing client is thread safe and all you
>>> should do is to create new bucket instance on every request
>>>
>>> pozdrawiam
>>> Paweł Kamiński
>>>
>>> kamiseq at gmail.com
>>> pkaminski.prv at gmail.com
>>> ______________________
>>>
>>>
>>>  On 10 October 2012 09:25, Pavel Kogan <pavel.kogan at cortica.com> wrote:
>>> > 1) Is it ok to share a single pbc client object between 50 threads?
>>> Should
>>> > it be protected by lock ?
>>> > 2) I didn't do load balancing between nodes yet, cause I want to
>>> understand
>>> > better throughput limit. I am planning to do it for much higher
>>> throughput.
>>> >
>>> > Pavel
>>> >
>>> >
>>> > On Wed, Oct 10, 2012 at 9:21 AM, kamiseq <kamiseq at gmail.com> wrote:
>>> >>
>>> >> maybe the good start is to share pbclient object and only create
>>> >> bucket per request, you will save few steps on client configuration.
>>> >> have you tried balancing requests to cluster and distribute them over
>>> all
>>> >> nodes?
>>> >>
>>> >> pozdrawiam
>>> >> Paweł Kamiński
>>> >>
>>> >> kamiseq at gmail.com
>>> >> pkaminski.prv at gmail.com
>>> >> ______________________
>>> >>
>>> >>
>>> >> On 10 October 2012 06:18, Pavel Kogan <pavel.kogan at cortica.com>
>>> wrote:
>>> >> > Hi all,
>>> >> >
>>> >> > I have Riak cluster consisting of 5 nodes that contains about 30
>>> >> > millions of
>>> >> > keys (35% of capacity according to Riak Control).
>>> >> > Currently we have single JAVA client reading and writing records to
>>> same
>>> >> > node. I need some tips, how to use the client efficiently
>>> >> > to reach maximal throughput - I would like to be able to read/write
>>> up
>>> >> > to
>>> >> > 100 records/sec on 1Gbit network. Currently I get a lot
>>> >> > of JAVA socket exceptions after a while (even for the much slower
>>> rate -
>>> >> > 10
>>> >> > records/sec), after which I  need to restart client and node.
>>> >> >
>>> >> > Thanks,
>>> >> >    Pavel
>>> >> >
>>> >> > P.S: My client using 50 threads and pbc client is created and
>>> >> > shut-downed
>>> >> > per request.
>>> >> >
>>> >> > _______________________________________________
>>> >> > riak-users mailing list
>>> >> > riak-users at lists.basho.com
>>> >> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>> >> >
>>> >
>>> >
>>>
>>
>>
>>
>> _______________________________________________
>> riak-users mailing listriak-users at lists.basho.comhttp://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20121010/bbbe3a6c/attachment.html>


More information about the riak-users mailing list