Java HTTP Client running out of FD's

Russell Brown russell.brown at mac.com
Thu Feb 24 15:37:05 EST 2011


On 24 Feb 2011, at 17:20, Abhishek Kona wrote:

>>>> 
>>> Each thread has a RiakClient.
>> RiakClient uses Apache HttpClient's MultiThreadedConnectionManager so
>> you could create a single RiakClient and share it among your threads
>> instead.
> Could you provide me an example (sample code / test case ) for that?
> In the ITestDataLoad.java file of the GitHub repo (http://goo.gl/Bu3gi), each thread seems to create a new RiakClient instead of sharing a single RiakClient, so am I missing something in following the same example.

Not really missing something, that _is_ how the test is coded, but it would be (marginally?) better to create the RiakClient once with the "right" number of threads. The HttpClient connection manager can pool the connections and reuse them, which you don't get if you bring a RiakClient into existence for a one request then lose it again. These tests are sort of artificial, but an application with (say) a service providing access to Riak could have a single RiakClient instance with the right number of threads and you can take advantage of connection persistence in HttpClient rather than creating and dropping and creating sockets.

Have a look at https://gist.github.com/842829 for an example of sharing a client across threads in ITestDataLoad

>> 
>>>> Is it possible to see the test code, maybe?
>>> I will try to post some snippets by tomorrow.
>> Cool, thanks. http://gist.github.com would be handy.
> https://gist.github.com/842472
> Here is a sample code, if you need more detail let me know. (there are a few internal libraries in use).

Thanks for that. I can't really see how it is being run, can the task be reused or does it do one request then get binned? Using a client for single request is not the best test though. Create a client, share it across threads and use it for repeated requests. I'll try and knock together and example that shows this vs. creating a new client per call.

I'll try and reproduce your issue at this end.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110224/6dcbcc16/attachment.html>


More information about the riak-users mailing list