Client get 503 response from riak server

Bora Kou bkou at zendesk.com
Wed Mar 4 14:22:45 EST 2015


Hi,

We have a 6 nodes riak cluster configured with 256 size.  We are using
versions 1.4.8. Occasionally, we see the server return 503 response.
Looking at the log, it seems that we have a lot of warning about reading
large object and busy_dist_port.  The majority of the objects are less than
2MB but we do have some large objects in the ranch of 5-15MB.

These are the two type of log entries we consistently see across all nodes
when that happen:

2015-03-04 17:53:48.149 [warning]
<0.1142.0>@riak_kv_vnode:do_get_object:1299 Reading large object of size
5498696 from...

2015-03-04 19:06:35.824 [info]
<0.95.0>@riak_core_sysmon_handler:handle_event:92 monitor busy_dist_port
<0.32259.1162>
[{initial_call,{riak_kv_get_fsm,init,1}},{almost_current_function,{gen_fsm,loop,7}},{message_queue_len,0}]
{#Port<0.3353>,'riak at servername...'}

The doc suggest increasing +zdbbl to deal with busy_dist_port.  But does
anybody have any other suggestion to what other parameters I should look at
to improve the situation?

Thank you,

-Bora


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20150304/1eb13b64/attachment-0002.html>


More information about the riak-users mailing list