NoNodesAvailableException

Ricardo Mayerhofer ricardo.ekm at gmail.com
Wed Feb 25 16:54:21 EST 2015


Hi all,
We're deploying a new application using Riak to store user cart during
purchase flow.

The application runs fine, however after a few hours all Riak operation
fails on the client side, even if the cluster is up and running ok.

The full stack exception is pasted at the end of this e-mail (
com.basho.riak.client.core.NoNodesAvailableException)

If the application is restarted it gets back working.

We're using Riak Client 2.0 along with Riak 1.4.10. We're using protocol
buffer with a TPC Load Balancer in front of Riak Cluster.

The load balancer has a Idle Period Time, so after that time it closes
connection (60 seconds).

It seem some sort of connection leak.

Any help is appreciated. Thanks

<25-02-2015 19:12:09> <Thread : http-nio-8080-exec-150>
<tid:CKT-66147652-c34e-44f8-858a-cc67bf183293> <customerid: > <[ERROR]
[com.b2winc.cart.riak.ShoppingCartRiakRepository] [Error]
---Stack : com.basho.riak.client.core.NoNodesAvailableException :
java.util.concurrent.ExecutionException at
com.basho.riak.client.core.FutureOperation.get(FutureOperation.java:260)
  at
com.basho.riak.client.api.commands.CoreFutureAdapter.get(CoreFutureAdapter.java:52)
  at com.basho.riak.client.api.RiakCommand.execute(RiakCommand.java:89)
  at com.basho.riak.client.api.RiakClient.execute(RiakClient.java:293)
  at
com.b2winc.cart.riak.ShoppingCartRiakRepository.isDependencyWorking(ShoppingCartRiakRepository.java:123)
  at
com.b2winc.cart.health.HealthService.getDependencies(HealthService.java:21)
  at
com.b2winc.cart.controller.HealthController.health(HealthController.java:24)
  at sun.reflect.GeneratedMethodAccessor474.invoke(Unknown Source)
  at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:483)
  at
org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:215)
  at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132)
  at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)
  at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:749)
  at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:689)
  at
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:83)
  at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:938)
  at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:870)
  at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:961)
  at
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:852)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:618)
  at
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:837)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
  at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
  at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
  at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
  at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at com.ocpsoft.pretty.PrettyFilter.doFilter(PrettyFilter.java:145)
  at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
  at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at
com.b2winc.checkout.web.ServerErrorFilter.doFilter(ServerErrorFilter.java:24)
  at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
  at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at
com.b2winc.checkout.web.TransactionIdFilter.doFilter(TransactionIdFilter.java:27)
  at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:344)
  at
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:261)
  at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
  at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
  at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
  at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
  at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
  at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
  at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:676)
  at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610)
  at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
  at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:534)
  at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1081)
  at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:658)
  at
org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222)
  at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1566)
  at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1523)
  at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
  at java.lang.Thread.run(Thread.java:745)
---Stack :  : com.basho.riak.client.core.NoNodesAvailableException  at
com.basho.riak.client.core.DefaultNodeManager.executeOnNode(DefaultNodeManager.java:95)
  at com.basho.riak.client.core.RiakCluster.execute(RiakCluster.java:197)
  at
com.basho.riak.client.core.RiakCluster.retryOperation(RiakCluster.java:328)
  at com.basho.riak.client.core.RiakCluster.access$800(RiakCluster.java:44)
  at
com.basho.riak.client.core.RiakCluster$RetryTask.run(RiakCluster.java:340)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
  at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
  at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  at java.lang.Thread.run(Thread.java:745)


We're using
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20150225/1fb59ffd/attachment-0002.html>


More information about the riak-users mailing list