Java HTTP Client running out of FD's

Abhishek Kona abhishek.kona at gmail.com
Thu Feb 24 23:06:59 EST 2011


On 25/02/11 2:07 AM, Russell Brown wrote:
>
> On 24 Feb 2011, at 17:20, Abhishek Kona wrote:
>
>>>>>
>>>> Each thread has a RiakClient.
>>> RiakClient uses Apache HttpClient's MultiThreadedConnectionManager so
>>> you could create a single RiakClient and share it among your threads
>>> instead.
>> Could you provide me an example (sample code / test case ) for that?
>> In the ITestDataLoad.java file of the GitHub repo 
>> (http://goo.gl/Bu3gi), each thread seems to create a new RiakClient 
>> instead of sharing a single RiakClient, so am I missing something in 
>> following the same example.
>
> Not really missing something, that _is_ how the test is coded, but it 
> would be (marginally?) better to create the RiakClient once with the 
> "right" number of threads. The HttpClient connection manager can pool 
> the connections and reuse them, which you don't get if you bring a 
> RiakClient into existence for a one request then lose it again. These 
> tests are sort of artificial, but an application with (say) a service 
> providing access to Riak could have a single RiakClient instance with 
> the right number of threads and you can take advantage of connection 
> persistence in HttpClient rather than creating and dropping and 
> creating sockets.
>
> Have a look at https://gist.github.com/842829 for an example of 
> sharing a client across threads in ITestDataLoad
>
>>>
>>>>> Is it possible to see the test code, maybe?
>>>> I will try to post some snippets by tomorrow.
>>> Cool, thanks. http://gist.github.com would be handy.
>> https://gist.github.com/842472
>> Here is a sample code, if you need more detail let me know. (there 
>> are a few internal libraries in use).
>
> Thanks for that. I can't really see how it is being run, can the task 
> be reused or does it do one request then get binned?
The task gets binned it is never reused.
> Using a client for single request is not the best test though. Create 
> a client, share it across threads and use it for repeated requests. 
> I'll try and knock together and example that shows this vs. creating a 
> new client per call
>
> I'll try and reproduce your issue at this end.
>
>
Thanks for it. I will try my tests by sharing a client.
-Abhishek Kona

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110225/4e1c1f5e/attachment.html>


More information about the riak-users mailing list