Java HTTP Client running out of FD's

Abhishek Kona abhishek.kona at gmail.com
Thu Feb 24 10:43:54 EST 2011


Hi Russel,

On 24/02/11 7:52 PM, Russell Brown wrote:
> On Thu, 2011-02-24 at 17:58 +0530, Abhishek Kona wrote:
>> Hi
>> Thanks for the quick reply.
>> On 24/02/11 5:03 PM, Russell Brown wrote:
>>> Hi Abhishek,
>>>
>>> On Thu, 2011-02-24 at 16:05 +0530, Abhishek Kona wrote:
>>>> Hi
>>>>
>>>> In our tests with RIAK the Java HTTP Client is running out of file
>>>> descriptors (currently 1024).
>>>> The test runs with 64 threads.
>>>>
>>>> Has this behavior been observed by anyone before?
>>>>
>>>> Any insights would be helpful.
>>> Can you let me know what platform you are running on, version of Java
>>> and which version of the client you are using, please?
>>>
>> We are currently running on a Debain Sid machine with Java  version
>> "1.6.0_22".
>>> Do you have the RiakConfig.setMaxConnections set to 64, too (or just
>>> your test ThreadPool)?
>>>
>> The test thread pool is set to 64 (each thread does a riak operation).
>
> Thanks for the info. Are you running a 64bit JVM?
>
We are running a 64 bit JVM.
> I'm wondering if you're sharing the same RiakClient between all threads
> or if each thread creates a RiakClient?
>
Each thread has a RiakClient.
> I'm also wondering what your RiakConfig.setMaxConnections setting is?
> (And which client version you have?)
>
This is not set, can I know what is the behavior in this case.
The client I am using is 0.9.1
> Is it possible to see the test code, maybe?
I will try to post some snippets by tomorrow.
>>> Also, just to sanity check, are you running against a Riak node on the
>>> same box?
>>>
>> The riak node was on the same box.
> http://wiki.basho.com/Open-Files-Limit.html says
>
> "Riak can consume a large number of open file handles during normal
> operation."
>
> Since your client and server are on the same machine, have you used lsof
> to check how many FDs Java (vs Riak itself) has and what state they are
> in, especially if any are hanging around after your test code has run?
>
I do not have the lsof count. I will post it as soon as I run into the 
same situation again.
> Sorry for the barrage of questions.
Thanks to you for looking into this.

-Abhishek Kona




More information about the riak-users mailing list