Riak JAVA Client Performance

Guido Medina guido.medina at temetra.com
Wed Oct 10 11:30:43 EDT 2012


In fact, as more nodes, you might be surprised it that it might be 
faster....see my point? Riak is a lot of things, 1st you have to be 
aware of the hashing, hashmap, how a key gets copied into different 
nodes, how one or more nodes are responsible for a key, etc...so it is 
not that simple.

On 10/10/12 16:28, Guido Medina wrote:
> That's why I keep pushing to one answer, Riak is not meant to be in 
> one cluster, you are removing the external factors and CAP settings 
> you will be using, and it won't be linear, you could get the same 
> results with RW=2 with 3, 4 and 5 nodes, there are several factors 
> that will influence your benchmark, I would start with 3 nodes, up to 
> 5 by altering those numbers, then you could end up with a formula 
> which I asure you, it won't be linear.
>
> Regards,
>
> Guido.
>
> On 10/10/12 16:19, Pavel Kogan wrote:
>> I understand that load balancing is a final solution, but I want to 
>> benchmark single node.
>> If I knew that I can load single node with N requests / sec, I could 
>> assume
>> that after load balancing over 5 nodes my throughput limit will 
>> increase linearly.
>>
>> Pavel
>>
>> On Wed, Oct 10, 2012 at 2:51 PM, Guido Medina 
>> <guido.medina at temetra.com <mailto:guido.medina at temetra.com>> wrote:
>>
>>     The answer is there, create a client config with N pooled
>>     connections to your load balancer whatever you are using, I know
>>     HA proxy supports the PBC config (TCP based) which is faster than
>>     HTTP client, and hence my recommendation.
>>
>>     Say, a non-clustered client config with N connections to
>>     balancer_host at 8087 and your balancer_host connected to EACH
>>     node, that's the way to go, the rest is about the CAP level you
>>     want to support which will impact your performance vs integrity.
>>     Up to you.
>>
>>     CAP doc:
>>     http://docs.basho.com/riak/latest/tutorials/fast-track/Tunable-CAP-Controls-in-Riak/
>>
>>     Guido.
>>
>>
>>     On 10/10/12 13:33, Pavel Kogan wrote:
>>>     Hi,
>>>
>>>     The node is OK and not down.
>>>     I have a way to do load balancing externally to JAVA Client.
>>>     I am evaluating Riak for using in my company and want to measure
>>>     maximal throughput vs single node.
>>>
>>>     Thanks,
>>>        Pavel
>>>
>>>     On Wed, Oct 10, 2012 at 2:13 PM, Guido Medina
>>>     <guido.medina at temetra.com <mailto:guido.medina at temetra.com>> wrote:
>>>
>>>         That question has been answered few times, here is my old
>>>         answer:
>>>
>>>         Hi,
>>>
>>>            It is the Java client which to be honest, doesn't handle well one node
>>>         going down, so, for example, in my company we use HA proxy for that, here is
>>>         a starting configuration:https://gist.github.com/1507077
>>>
>>>            Once we switched to HA proxy we just use a simple client without cluster
>>>         config, so the Java client doesn't know anything about the load balancing
>>>         going on. It works well, I can upgrade and restart servers without our Java
>>>         application be complaining.
>>>
>>>         Regards,
>>>
>>>         Guido.
>>>
>>>
>>>         On 10/10/12 12:58, Pavel Kogan wrote:
>>>>         Thanks,
>>>>
>>>>         I will try this solution.
>>>>
>>>>         Pavel
>>>>
>>>>         On Wed, Oct 10, 2012 at 1:51 PM, kamiseq <kamiseq at gmail.com
>>>>         <mailto:kamiseq at gmail.com>> wrote:
>>>>
>>>>             well I asked same question few days ago (maybe 2 weeks
>>>>             form now) and
>>>>             the answer was that yes sharing client is thread safe
>>>>             and all you
>>>>             should do is to create new bucket instance on every request
>>>>
>>>>             pozdrawiam
>>>>             Paweł Kamiński
>>>>
>>>>             kamiseq at gmail.com <mailto:kamiseq at gmail.com>
>>>>             pkaminski.prv at gmail.com <mailto:pkaminski.prv at gmail.com>
>>>>             ______________________
>>>>
>>>>
>>>>             On 10 October 2012 09:25, Pavel Kogan
>>>>             <pavel.kogan at cortica.com
>>>>             <mailto:pavel.kogan at cortica.com>> wrote:
>>>>             > 1) Is it ok to share a single pbc client object
>>>>             between 50 threads? Should
>>>>             > it be protected by lock ?
>>>>             > 2) I didn't do load balancing between nodes yet,
>>>>             cause I want to understand
>>>>             > better throughput limit. I am planning to do it for
>>>>             much higher throughput.
>>>>             >
>>>>             > Pavel
>>>>             >
>>>>             >
>>>>             > On Wed, Oct 10, 2012 at 9:21 AM, kamiseq
>>>>             <kamiseq at gmail.com <mailto:kamiseq at gmail.com>> wrote:
>>>>             >>
>>>>             >> maybe the good start is to share pbclient object and
>>>>             only create
>>>>             >> bucket per request, you will save few steps on
>>>>             client configuration.
>>>>             >> have you tried balancing requests to cluster and
>>>>             distribute them over all
>>>>             >> nodes?
>>>>             >>
>>>>             >> pozdrawiam
>>>>             >> Paweł Kamiński
>>>>             >>
>>>>             >> kamiseq at gmail.com <mailto:kamiseq at gmail.com>
>>>>             >> pkaminski.prv at gmail.com <mailto:pkaminski.prv at gmail.com>
>>>>             >> ______________________
>>>>             >>
>>>>             >>
>>>>             >> On 10 October 2012 06:18, Pavel Kogan
>>>>             <pavel.kogan at cortica.com
>>>>             <mailto:pavel.kogan at cortica.com>> wrote:
>>>>             >> > Hi all,
>>>>             >> >
>>>>             >> > I have Riak cluster consisting of 5 nodes that
>>>>             contains about 30
>>>>             >> > millions of
>>>>             >> > keys (35% of capacity according to Riak Control).
>>>>             >> > Currently we have single JAVA client reading and
>>>>             writing records to same
>>>>             >> > node. I need some tips, how to use the client
>>>>             efficiently
>>>>             >> > to reach maximal throughput - I would like to be
>>>>             able to read/write up
>>>>             >> > to
>>>>             >> > 100 records/sec on 1Gbit network. Currently I get
>>>>             a lot
>>>>             >> > of JAVA socket exceptions after a while (even for
>>>>             the much slower rate -
>>>>             >> > 10
>>>>             >> > records/sec), after which I  need to restart
>>>>             client and node.
>>>>             >> >
>>>>             >> > Thanks,
>>>>             >> >  Pavel
>>>>             >> >
>>>>             >> > P.S: My client using 50 threads and pbc client is
>>>>             created and
>>>>             >> > shut-downed
>>>>             >> > per request.
>>>>             >> >
>>>>             >> > _______________________________________________
>>>>             >> > riak-users mailing list
>>>>             >> > riak-users at lists.basho.com
>>>>             <mailto:riak-users at lists.basho.com>
>>>>             >> >
>>>>             http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>             >> >
>>>>             >
>>>>             >
>>>>
>>>>
>>>>
>>>>
>>>>         _______________________________________________
>>>>         riak-users mailing list
>>>>         riak-users at lists.basho.com  <mailto:riak-users at lists.basho.com>
>>>>         http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>>
>>>         _______________________________________________
>>>         riak-users mailing list
>>>         riak-users at lists.basho.com <mailto:riak-users at lists.basho.com>
>>>         http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>>
>>
>>
>>     _______________________________________________
>>     riak-users mailing list
>>     riak-users at lists.basho.com <mailto:riak-users at lists.basho.com>
>>     http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20121010/ccd95010/attachment.html>


More information about the riak-users mailing list