Java HTTP Client running out of FD's

Russell Brown russell.brown at mac.com
Thu Feb 24 09:22:54 EST 2011


On Thu, 2011-02-24 at 17:58 +0530, Abhishek Kona wrote:
> Hi
> Thanks for the quick reply.
> On 24/02/11 5:03 PM, Russell Brown wrote:
> > Hi Abhishek,
> >
> > On Thu, 2011-02-24 at 16:05 +0530, Abhishek Kona wrote:
> >> Hi
> >>
> >> In our tests with RIAK the Java HTTP Client is running out of file
> >> descriptors (currently 1024).
> >> The test runs with 64 threads.
> >>
> >> Has this behavior been observed by anyone before?
> >>
> >> Any insights would be helpful.
> > Can you let me know what platform you are running on, version of Java
> > and which version of the client you are using, please?
> >
> We are currently running on a Debain Sid machine with Java  version 
> "1.6.0_22".
> > Do you have the RiakConfig.setMaxConnections set to 64, too (or just
> > your test ThreadPool)?
> >
> The test thread pool is set to 64 (each thread does a riak operation).


Thanks for the info. Are you running a 64bit JVM? 

I'm wondering if you're sharing the same RiakClient between all threads
or if each thread creates a RiakClient?

I'm also wondering what your RiakConfig.setMaxConnections setting is?
(And which client version you have?)

Is it possible to see the test code, maybe?

> > Also, just to sanity check, are you running against a Riak node on the
> > same box?
> >
> The riak node was on the same box.

http://wiki.basho.com/Open-Files-Limit.html says

"Riak can consume a large number of open file handles during normal
operation."

Since your client and server are on the same machine, have you used lsof
to check how many FDs Java (vs Riak itself) has and what state they are
in, especially if any are hanging around after your test code has run? 

Sorry for the barrage of questions.






More information about the riak-users mailing list