2i timeouts in 1.4

Sean McKibben graphex at graphex.com
Fri Jul 26 19:00:13 EDT 2013


Thank you for looking in to this. This is a major problem for our production cluster, and we're in a bit of a bind right now trying to figure out a workaround in the interim. It sounds like maybe a mapreduce might handle the timeout properly, so hopefully we can do that in the meantime.
If there is any way we can have a hotfix ASAP though, that would be preferable. Certainly would not be a problem for us to edit a value in the config file (and given the lack of support in the ruby client for the timeout setting, the ability to edit the global default would be preferred).
In the ruby client i had to monkeypatch it like this to even submit the timeout value, which is not ideal:

module Riak
  class Client
    class HTTPBackend
      def get_index(bucket, index, query)
        bucket = bucket.name if Bucket === bucket
        path = case query
               when Range
                 raise ArgumentError, t('invalid_index_query', :value => query.inspect) unless String === query.begin || Integer === query.end
                 index_range_path(bucket, index, query.begin, query.end)
               when String, Integer
                 index_eq_path(bucket, index, query, 'timeout' => '260000')
               else
                 raise ArgumentError, t('invalid_index_query', :value => query.inspect)
               end
        response = get(200, path)
        JSON.parse(response[:body])['keys']
      end
    end
  end
end

Thanks for the update,
Sean



On Jul 26, 2013, at 4:49 PM, Russell Brown <russell.brown at mac.com> wrote:

> Hi Sean,
> I'm very sorry to say that you've found a featurebug.
> 
> There was a fix put in here https://github.com/basho/riak_core/pull/332
> 
> But that means that the default timeout of 60 seconds is now honoured. In the past it was not.
> 
> As far as I can see the 2i endpoint never accepted a timeout argument, and it still does not.
> 
> The fix would be to add the timeout to the 2i API endpoints, and I'll do that straight away.
> 
> In the meantime, I wonder if streaming the results would help, or if you'd still hit the overall timeout?
> 
> Very sorry that you've run into this. Let me know if streaming helps, I've raised an issue here[1] if you want to track this bug
> 
> Cheers
> 
> Russell
> 
> [1] https://github.com/basho/riak_kv/issues/610
> 
> 
> On 26 Jul 2013, at 17:59, Sean McKibben <graphex at graphex.com> wrote:
> 
>> I should have mentioned that I also tried:
>> curl -H "X-Riak-Timeout: 260000" "http://127.0.0.1:8098/buckets/mybucket/index/test_bin/myval?timeout=260000" -i
>> but still receive the 500 error below exactly at the 60 second mark. Is this a bug?
>> 
>> Secondary to getting this working at all, is this documented anywhere? and any way to set this timeout using the ruby riak client?
>> 
>> Stream may well work, but I'm going to have to make a number of changes on the client side to deal with the results.
>> 
>> Sean
>> 
>> On Jul 26, 2013, at 3:53 PM, Brian Roach <roach at basho.com> wrote:
>> 
>>> Sean -
>>> 
>>> The timeout isn't via a header, it's a query param -> &timeout=xxxx
>>> 
>>> You can also use stream=true to stream the results.
>>> 
>>> - Roach
>>> 
>>> Sent from my iPhone
>>> 
>>> On Jul 26, 2013, at 3:43 PM, Sean McKibben <graphex at graphex.com> wrote:
>>> 
>>>> We just upgraded to 1.4 and are having a big problem with some of our larger 2i queries. We have a few key queries that takes longer than 60 seconds (usually about 110 seconds) to execute, but after going to 1.4 we can't seem to get around a 60 second timeout.
>>>> 
>>>> I've tried:
>>>> curl -H "X-Riak-Timeout: 260000" "http://127.0.0.1:8098/buckets/mybucket/index/test_bin/myval?x-riak-timeout=260000" -i
>>>> 
>>>> But I always get
>>>> HTTP/1.1 500 Internal Server Error
>>>> Vary: Accept-Encoding
>>>> Server: MochiWeb/1.1 WebMachine/1.10.0 (never breaks eye contact)
>>>> Date: Fri, 26 Jul 2013 21:41:28 GMT
>>>> Content-Type: text/html
>>>> Content-Length: 265
>>>> Connection: close
>>>> 
>>>> <html><head><title>500 Internal Server Error</title></head><body><h1>Internal Server Error</h1>The server encountered an error while processing this request:<br><pre>{error,{error,timeout}}</pre><P><HR><ADDRESS>mochiweb+webmachine web server</ADDRESS></body></html>
>>>> 
>>>> Right at the 60 second mark. What can I set to give my secondary index queries more time??
>>>> 
>>>> This is causing major problems for us :(
>>>> 
>>>> Sean
>>>> _______________________________________________
>>>> riak-users mailing list
>>>> riak-users at lists.basho.com
>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> 
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> 





More information about the riak-users mailing list