Java HTTP Client running out of FD's

Abhishek Kona abhishek.kona at gmail.com
Thu Feb 24 23:28:57 EST 2011


Hi Russell,

On 25/02/11 9:36 AM, Abhishek Kona wrote:
> On 25/02/11 2:07 AM, Russell Brown wrote:
>>
>> On 24 Feb 2011, at 17:20, Abhishek Kona wrote:
>>
>>>>>>
>>>>> Each thread has a RiakClient.
>>>> RiakClient uses Apache HttpClient's MultiThreadedConnectionManager so
>>>> you could create a single RiakClient and share it among your threads
>>>> instead.
>>> Could you provide me an example (sample code / test case ) for that?
>>> In the ITestDataLoad.java file of the GitHub repo 
>>> (http://goo.gl/Bu3gi), each thread seems to create a new RiakClient 
>>> instead of sharing a single RiakClient, so am I missing something in 
>>> following the same example.
>>
>> Not really missing something, that _is_ how the test is coded, but it 
>> would be (marginally?) better to create the RiakClient once with the 
>> "right" number of threads. The HttpClient connection manager can pool 
>> the connections and reuse them, which you don't get if you bring a 
>> RiakClient into existence for a one request then lose it again. These 
>> tests are sort of artificial, but an application with (say) a service 
>> providing access to Riak could have a single RiakClient instance with 
>> the right number of threads and you can take advantage of connection 
>> persistence in HttpClient rather than creating and dropping and 
>> creating sockets.
>>
>> Have a look at https://gist.github.com/842829 for an example of 
>> sharing a client across threads in ITestDataLoad
>>
>>>>
>>>>>> Is it possible to see the test code, maybe?
>>>>> I will try to post some snippets by tomorrow.
>>>> Cool, thanks. http://gist.github.com would be handy.
>>> https://gist.github.com/842472
>>> Here is a sample code, if you need more detail let me know. (there 
>>> are a few internal libraries in use).
>>
>> Thanks for that. I can't really see how it is being run, can the task 
>> be reused or does it do one request then get binned?
> The task gets binned it is never reused.
>> Using a client for single request is not the best test though. Create 
>> a client, share it across threads and use it for repeated requests. 
>> I'll try and knock together and example that shows this vs. creating 
>> a new client per call
>>

Also I am not clear about the RiakClient's thread safety. If I share the 
RiakClient should the client code maintain locks on it.

>> I'll try and reproduce your issue at this end.
>>
>>
> Thanks for it. I will try my tests by sharing a client.
> -Abhishek Kona
>
-Abhishek Kona

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110225/5953a2a7/attachment.html>


More information about the riak-users mailing list