Java HTTP Client running out of FD's

Russell Brown russell.brown at mac.com
Thu Feb 24 11:51:34 EST 2011


Thanks for getting back to me.

On Thu, 2011-02-24 at 21:13 +0530, Abhishek Kona wrote:

> > I'm wondering if you're sharing the same RiakClient between all threads
> > or if each thread creates a RiakClient?
> >
> Each thread has a RiakClient.

RiakClient uses Apache HttpClient's MultiThreadedConnectionManager so
you could create a single RiakClient and share it among your threads
instead.

> > I'm also wondering what your RiakConfig.setMaxConnections setting is?
> > (And which client version you have?)
> >
> This is not set, can I know what is the behavior in this case.

This is how many concurrent connections HttpClient can create. So for
your 64 threads you could create a RiakConfig with 				
    riakConfig.setMaxConnections(64); 


> The client I am using is 0.9.1

The version you are using will limit the maximum connections per host to
2, you can get the latest from

http://github.com/basho/riak-java-client 

Or if you need to stick with your current version you can pass an
HttpClient instance configured for 64 connections to your RiakConfig (eg
https://gist.github.com/842356)

> > Is it possible to see the test code, maybe?
> I will try to post some snippets by tomorrow.

Cool, thanks. http://gist.github.com would be handy.

> >>> Also, just to sanity check, are you running against a Riak node on the
> >>> same box?
> >>>
> >> The riak node was on the same box.
> > http://wiki.basho.com/Open-Files-Limit.html says
> >
> > "Riak can consume a large number of open file handles during normal
> > operation."
> >
> > Since your client and server are on the same machine, have you used lsof
> > to check how many FDs Java (vs Riak itself) has and what state they are
> > in, especially if any are hanging around after your test code has run?
> >
> I do not have the lsof count. I will post it as soon as I run into the 
> same situation again.

Appreciate it.

> > Sorry for the barrage of questions.
> Thanks to you for looking into this.
> 
> -Abhishek Kona






More information about the riak-users mailing list