Java HTTP Client running out of FD's

Abhishek Kona abhishek.kona at gmail.com
Thu Feb 24 12:20:54 EST 2011


On 24/02/11 10:21 PM, Russell Brown wrote:
> Thanks for getting back to me.
>
> On Thu, 2011-02-24 at 21:13 +0530, Abhishek Kona wrote:
>
>>> I'm wondering if you're sharing the same RiakClient between all threads
>>> or if each thread creates a RiakClient?
>>>
>> Each thread has a RiakClient.
> RiakClient uses Apache HttpClient's MultiThreadedConnectionManager so
> you could create a single RiakClient and share it among your threads
> instead.
Could you provide me an example (sample code / test case ) for that?
In the ITestDataLoad.java file of the GitHub repo (http://goo.gl/Bu3gi), 
each thread seems to create a new RiakClient instead of sharing a single 
RiakClient, so am I missing something in following the same example.
>>> I'm also wondering what your RiakConfig.setMaxConnections setting is?
>>> (And which client version you have?)
>>>
>> This is not set, can I know what is the behavior in this case.
> This is how many concurrent connections HttpClient can create. So for
> your 64 threads you could create a RiakConfig with 				
>      riakConfig.setMaxConnections(64);
>
>
>> The client I am using is 0.9.1
> The version you are using will limit the maximum connections per host to
> 2, you can get the latest from
>
> http://github.com/basho/riak-java-client
>
> Or if you need to stick with your current version you can pass an
> HttpClient instance configured for 64 connections to your RiakConfig (eg
> https://gist.github.com/842356)
>
>>> Is it possible to see the test code, maybe?
>> I will try to post some snippets by tomorrow.
> Cool, thanks. http://gist.github.com would be handy.
https://gist.github.com/842472
Here is a sample code, if you need more detail let me know. (there are a 
few internal libraries in use).
>>>>> Also, just to sanity check, are you running against a Riak node on the
>>>>> same box?
>>>>>
>>>> The riak node was on the same box.
>>> http://wiki.basho.com/Open-Files-Limit.html says
>>>
>>> "Riak can consume a large number of open file handles during normal
>>> operation."
>>>
>>> Since your client and server are on the same machine, have you used lsof
>>> to check how many FDs Java (vs Riak itself) has and what state they are
>>> in, especially if any are hanging around after your test code has run?
>>>
>> I do not have the lsof count. I will post it as soon as I run into the
>> same situation again.
> Appreciate it.
>
>>> Sorry for the barrage of questions.
>> Thanks to you for looking into this.
>>
>> -Abhishek Kona
>





More information about the riak-users mailing list