occasional timeout when deleting key on multi-node Riak 1.4

Dmitri Zagidulin dzagidulin at basho.com
Thu Oct 2 12:20:55 EDT 2014


Thanks. Are there entries in any of the other logs? (like the crash dump).

Can you also provide more info on the nodes themselves. What size AWS
instances are you running? Is the delete timeout happening while load
testing?

On Thu, Oct 2, 2014 at 12:11 PM, Igor Senderovich <
isenderovich at esperdyne.com> wrote:

> Thanks for your help, Dmitri,
>
> I get the following in error.log:
> 2014-10-02 12:05:45.037 [error] <0.6359.19> Webmachine error at path
> "/buckets/imc/keys/5134a18660494ea5553d2c90ef9eea2f" : "Service Unavailable"
>
> And no, there is no load balancer on our cluster.
> Thank you
>
>
> On Thu, Oct 2, 2014 at 11:52 AM, Dmitri Zagidulin <dzagidulin at basho.com>
> wrote:
>
>> One other question - are you using a load balancer for your cluster (like
>> HAProxy or the like). In which case, take a look at its logs, also.
>>
>> On Thu, Oct 2, 2014 at 11:51 AM, Dmitri Zagidulin <dzagidulin at basho.com>
>> wrote:
>>
>>> Igor,
>>> Can you look in the riak log directory, in the error.log (and console
>>> log and crash dump file) to see if there's any entries, around the time of
>>> the delete operation? And post them here?
>>>
>>>
>>>
>>> On Thu, Oct 2, 2014 at 11:45 AM, Igor Senderovich <
>>> isenderovich at esperdyne.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I get a timeout when deleting a key, reproducible in about 1 in 10
>>>> times:
>>>> $ curl -i -vvv
>>>> http://myhost:8098/buckets/imc/keys/5134a18660494ea5553d2c90ef9eea2f
>>>>
>>>> * About to connect() to dp1.prod6.ec2.cmg.net port 8098
>>>> *   Trying 10.12.239.90... connected
>>>> * Connected to dp1.prod6.ec2.cmg.net (10.12.239.90) port 8098
>>>> > DELETE /buckets/imc/keys/5134a18660494ea5553d2c90ef9eea2f HTTP/1.1
>>>> > User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5
>>>> OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
>>>> > Host: dp1.prod6.ec2.cmg.net:8098
>>>> > Accept: */*
>>>> >
>>>> < HTTP/1.1 503 Service Unavailable
>>>> HTTP/1.1 503 Service Unavailable
>>>> < Server: MochiWeb/1.1 WebMachine/1.10.0 (never breaks eye contact)
>>>> Server: MochiWeb/1.1 WebMachine/1.10.0 (never breaks eye contact)
>>>> < Date: Wed, 01 Oct 2014 16:11:41 GMT
>>>> Date: Wed, 01 Oct 2014 16:11:41 GMT
>>>> < Content-Type: text/plain
>>>> Content-Type: text/plain
>>>> < Content-Length: 18
>>>> Content-Length: 18
>>>>
>>>> request timed out
>>>> * Connection #0 to host dp1.prod6.ec2.cmg.net left intact
>>>> * Closing connection #0
>>>>
>>>>
>>>> This is on Riak 1.4 on a 5 node cluster with an n-value of 3.
>>>> Thank you for your help
>>>>
>>>> _______________________________________________
>>>> riak-users mailing list
>>>> riak-users at lists.basho.com
>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20141002/d0f80fda/attachment.html>


More information about the riak-users mailing list