Multiget java client performance

Brian Roach roach at basho.com
Tue Jan 20 12:13:56 EST 2015


Santi -

The core of the 2.0 Java client uses Netty. There's a fixed number of
worker threads that process sockets using non-blocking IO ( w/
polling/select).

Due to the synchronous nature of the Riak API, each fetch operation
requires its own socket connection and we can't pipeline.

As noted in the Javadoc for MultiFetch
(http://basho.github.io/riak-java-client/2.0.0/com/basho/riak/client/api/commands/kv/MultiFetch.html)
there is a default of 10 simultaneous fetches in flight at once. This
is pretty conservative but ... that's what defaults are for :)

How does it scale? Depends on the hardware, network latency, etc. In
the end ... you've only got so many threads and you prob don't want
1000 sockets being created at once. Best suggestion is to adjust up
the number of simultaneous inflight and measure it.

Thanks,
- Roach



On Mon, Jan 19, 2015 at 6:43 PM, Santi Kumar <santi at veradocs.com> wrote:
> Brain
> I"m using Riak Client 2.0.0 and Riak 2.0.2.
>
>
>
> On Tue, Jan 20, 2015 at 1:54 AM, Brian Roach <roach at basho.com> wrote:
>>
>> Santi -
>>
>> Which version of the Java client?
>>
>> Thanks,
>> - Roach
>>
>> On Mon, Jan 19, 2015 at 7:36 AM, Santi Kumar <santi at veradocs.com> wrote:
>> > Hi
>> >
>> > We are using java client for accessing Riak KV/Search. For some use
>> > cases,
>> > we go to search, get the keys and access the data from Riak. There might
>> > be
>> > a case where we might get 1000's of keys. So want to understand what is
>> > the
>> > impact of that on multiget and how does it scale.
>> >
>> >
>> > We were using RDBMS and Elastic search earlier. Now we moved all the
>> > data to
>> > Riak KV and Search. We used to query all the audit entries from ES as we
>> > used to store the complete data there. Applicaitons flows used to get
>> > the
>> > data from RDBMS. Now as we replaced everything with Riak KV and search,
>> > We
>> > need to go to Riak Search for auditing reports. If we store all the
>> > objects
>> > in Riak Search, how does it impact ?
>> >
>> > Thanks
>> > Santi
>> >
>> > _______________________________________________
>> > riak-users mailing list
>> > riak-users at lists.basho.com
>> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> >
>
>




More information about the riak-users mailing list