NoNodesAvailableException

Brian Roach roach at basho.com
Wed Feb 25 17:21:36 EST 2015


Ricardo,

That exception gets raised by the DefaultNodeManager when all RiakNode
instances are reporting that no connections are available from the
pool - e.g. a max number of connections have been defined and they're
all out of the pool and in use.

Not sure why that would be happening; if connections get closed by the
remote peer the socket is discarded and the permit is returned to the
pool's controlling semaphore.

Could you enable logging in the client (set to debug)? The RiakNode
will spit out a ton of info that should let us see what's going on
with the connections.

(The client uses SLF4J so you just need to configure a logger in the
dependencies)

Thanks,
- Roach


On Wed, Feb 25, 2015 at 2:54 PM, Ricardo Mayerhofer
<ricardo.ekm at gmail.com> wrote:
> Hi all,
> We're deploying a new application using Riak to store user cart during
> purchase flow.
>
> The application runs fine, however after a few hours all Riak operation
> fails on the client side, even if the cluster is up and running ok.
>
> The full stack exception is pasted at the end of this e-mail
> (com.basho.riak.client.core.NoNodesAvailableException)
>
> If the application is restarted it gets back working.
>
> We're using Riak Client 2.0 along with Riak 1.4.10. We're using protocol
> buffer with a TPC Load Balancer in front of Riak Cluster.
>
> The load balancer has a Idle Period Time, so after that time it closes
> connection (60 seconds).
>
> It seem some sort of connection leak.
>
> Any help is appreciated. Thanks
>
> <25-02-2015 19:12:09> <Thread : http-nio-8080-exec-150>
> <tid:CKT-66147652-c34e-44f8-858a-cc67bf183293> <customerid: > <[ERROR]
> [com.b2winc.cart.riak.ShoppingCartRiakRepository] [Error]
> ---Stack : com.basho.riak.client.core.NoNodesAvailableException :
> java.util.concurrent.ExecutionException at
> com.basho.riak.client.core.FutureOperation.get(FutureOperation.java:260)
>   at
> com.basho.riak.client.api.commands.CoreFutureAdapter.get(CoreFutureAdapter.java:52)
>   at com.basho.riak.client.api.RiakCommand.execute(RiakCommand.java:89)
>   at com.basho.riak.client.api.RiakClient.execute(RiakClient.java:293)
>   at
> com.b2winc.cart.riak.ShoppingCartRiakRepository.isDependencyWorking(ShoppingCartRiakRepository.java:123)
>   at
> com.b2winc.cart.health.HealthService.getDependencies(HealthService.java:21)
>   at
> com.b2winc.cart.controller.HealthController.health(HealthController.java:24)
>   at sun.reflect.GeneratedMethodAccessor474.invoke(Unknown Source)
>   at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:483)
>   at
> org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:215)
>   at
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132)
>   at
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)
>   at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:749)
>   at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:689)
>   at
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:83)
>   at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:938)
>   at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:870)
>   at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:961)
>   at
> org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:852)
>   at javax.servlet.http.HttpServlet.service(HttpServlet.java:618)
>   at
> org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:837)
>   at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
>   at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
>   at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>   at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
>   at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
>   at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>   at com.ocpsoft.pretty.PrettyFilter.doFilter(PrettyFilter.java:145)
>   at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
>   at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>   at
> com.b2winc.checkout.web.ServerErrorFilter.doFilter(ServerErrorFilter.java:24)
>   at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
>   at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>   at
> com.b2winc.checkout.web.TransactionIdFilter.doFilter(TransactionIdFilter.java:27)
>   at
> org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:344)
>   at
> org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:261)
>   at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
>   at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>   at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
>   at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
>   at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
>   at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
>   at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
>   at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:676)
>   at
> org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610)
>   at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
>   at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:534)
>   at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1081)
>   at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:658)
>   at
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222)
>   at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1566)
>   at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1523)
>   at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
>   at java.lang.Thread.run(Thread.java:745)
> ---Stack :  : com.basho.riak.client.core.NoNodesAvailableException  at
> com.basho.riak.client.core.DefaultNodeManager.executeOnNode(DefaultNodeManager.java:95)
>   at com.basho.riak.client.core.RiakCluster.execute(RiakCluster.java:197)
>   at
> com.basho.riak.client.core.RiakCluster.retryOperation(RiakCluster.java:328)
>   at com.basho.riak.client.core.RiakCluster.access$800(RiakCluster.java:44)
>   at
> com.basho.riak.client.core.RiakCluster$RetryTask.run(RiakCluster.java:340)
>   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>   at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>   at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
>
>
> We're using
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>




More information about the riak-users mailing list