Java HTTP Client running out of FD's
abhishek.kona at gmail.com
Thu Feb 24 12:20:54 EST 2011
On 24/02/11 10:21 PM, Russell Brown wrote:
> Thanks for getting back to me.
> On Thu, 2011-02-24 at 21:13 +0530, Abhishek Kona wrote:
>>> I'm wondering if you're sharing the same RiakClient between all threads
>>> or if each thread creates a RiakClient?
>> Each thread has a RiakClient.
> RiakClient uses Apache HttpClient's MultiThreadedConnectionManager so
> you could create a single RiakClient and share it among your threads
Could you provide me an example (sample code / test case ) for that?
In the ITestDataLoad.java file of the GitHub repo (http://goo.gl/Bu3gi),
each thread seems to create a new RiakClient instead of sharing a single
RiakClient, so am I missing something in following the same example.
>>> I'm also wondering what your RiakConfig.setMaxConnections setting is?
>>> (And which client version you have?)
>> This is not set, can I know what is the behavior in this case.
> This is how many concurrent connections HttpClient can create. So for
> your 64 threads you could create a RiakConfig with
>> The client I am using is 0.9.1
> The version you are using will limit the maximum connections per host to
> 2, you can get the latest from
> Or if you need to stick with your current version you can pass an
> HttpClient instance configured for 64 connections to your RiakConfig (eg
>>> Is it possible to see the test code, maybe?
>> I will try to post some snippets by tomorrow.
> Cool, thanks. http://gist.github.com would be handy.
Here is a sample code, if you need more detail let me know. (there are a
few internal libraries in use).
>>>>> Also, just to sanity check, are you running against a Riak node on the
>>>>> same box?
>>>> The riak node was on the same box.
>>> http://wiki.basho.com/Open-Files-Limit.html says
>>> "Riak can consume a large number of open file handles during normal
>>> Since your client and server are on the same machine, have you used lsof
>>> to check how many FDs Java (vs Riak itself) has and what state they are
>>> in, especially if any are hanging around after your test code has run?
>> I do not have the lsof count. I will post it as soon as I run into the
>> same situation again.
> Appreciate it.
>>> Sorry for the barrage of questions.
>> Thanks to you for looking into this.
>> -Abhishek Kona
More information about the riak-users