Whole cluster times out if one node is gone
j4yferd at gmail.com
Mon Nov 22 20:59:51 EST 2010
Thanks for the response! I tried it again while watching `riak-admin
status` - basically, it takes about 30 seconds of node C being down before
riak realizes it's gone. During that time, if I'm writing to the cluster at
all (I throttled it to 2 writes per second for testing), both writes and
reads hang indefinitely, and sometimes time out.
I'm using Ripple to do the writes, and wget to test reads, all on node A for
now, since I know it'll be up. I'm using the default R and W options for
Thanks for the help and clarification around ringready.
On Mon, Nov 22, 2010 at 5:15 PM, Dan Reverri <dan at basho.com> wrote:
> Your HTTP calls should not being timing out. Are you sending requests
> directly to the Riak node or are you using a load balancer? How much load
> are you placing on node A? Is it a write only load or are there reads as
> well? Can you confirm "all" requests time out or is it a large subset of the
> requests? How large are the objects being written? Are you setting R and W
> in the request? Are you using a particular client (Ruby, Python, etc.)? Can
> you provide the output of "riak-admin status" from node A?
> Regarding the ringready command; that is behaving as I would expect
> considering a node is down.
> Daniel Reverri
> Developer Advocate
> Basho Technologies, Inc.
> dan at basho.com
> On Mon, Nov 22, 2010 at 4:55 PM, Jay Adkisson <j4yferd at gmail.com> wrote:
>> Hey all,
>> Here's what I'm seeing: I have four nodes A, B, C, and D. I'm loading
>> lots of data into node A, which is being distributed evenly across the
>> nodes. If I physically reboot node D, all my HTTP calls time out, and
>> `riak-admin ringready` complains that not all nodes are up. Is this
>> intended behavior? Is there a configuration option I can set so it fails
>> more gracefully?
>> riak-users mailing list
>> riak-users at lists.basho.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the riak-users