2i timeouts in 1.4

Russell Brown russell.brown at mac.com
Fri Jul 26 20:43:21 EDT 2013

For a work around you could use streaming and pagination.

Request smaller pages of data (i.e. sub 60 seconds worth) and use streaming to get the results to your client sooner.

In HTTP this would look like

your results will include a continuation like


and you can use that to get the next N results. Breaking your query up that way should duck the timeout.

Furthermore, adding &stream=true will mean the first results is received very rapidly.

I don't think the Ruby client is up to date for the new 2i features, but you could monkeypatch as before.



On 26 Jul 2013, at 19:00, Sean McKibben <graphex at graphex.com> wrote:

> Thank you for looking in to this. This is a major problem for our production cluster, and we're in a bit of a bind right now trying to figure out a workaround in the interim. It sounds like maybe a mapreduce might handle the timeout properly, so hopefully we can do that in the meantime.
> If there is any way we can have a hotfix ASAP though, that would be preferable. Certainly would not be a problem for us to edit a value in the config file (and given the lack of support in the ruby client for the timeout setting, the ability to edit the global default would be preferred).
> In the ruby client i had to monkeypatch it like this to even submit the timeout value, which is not ideal:
> module Riak
>  class Client
>    class HTTPBackend
>      def get_index(bucket, index, query)
>        bucket = bucket.name if Bucket === bucket
>        path = case query
>               when Range
>                 raise ArgumentError, t('invalid_index_query', :value => query.inspect) unless String === query.begin || Integer === query.end
>                 index_range_path(bucket, index, query.begin, query.end)
>               when String, Integer
>                 index_eq_path(bucket, index, query, 'timeout' => '260000')
>               else
>                 raise ArgumentError, t('invalid_index_query', :value => query.inspect)
>               end
>        response = get(200, path)
>        JSON.parse(response[:body])['keys']
>      end
>    end
>  end
> end
> Thanks for the update,
> Sean
> On Jul 26, 2013, at 4:49 PM, Russell Brown <russell.brown at mac.com> wrote:
>> Hi Sean,
>> I'm very sorry to say that you've found a featurebug.
>> There was a fix put in here https://github.com/basho/riak_core/pull/332
>> But that means that the default timeout of 60 seconds is now honoured. In the past it was not.
>> As far as I can see the 2i endpoint never accepted a timeout argument, and it still does not.
>> The fix would be to add the timeout to the 2i API endpoints, and I'll do that straight away.
>> In the meantime, I wonder if streaming the results would help, or if you'd still hit the overall timeout?
>> Very sorry that you've run into this. Let me know if streaming helps, I've raised an issue here[1] if you want to track this bug
>> Cheers
>> Russell
>> [1] https://github.com/basho/riak_kv/issues/610
>> On 26 Jul 2013, at 17:59, Sean McKibben <graphex at graphex.com> wrote:
>>> I should have mentioned that I also tried:
>>> curl -H "X-Riak-Timeout: 260000" "" -i
>>> but still receive the 500 error below exactly at the 60 second mark. Is this a bug?
>>> Secondary to getting this working at all, is this documented anywhere? and any way to set this timeout using the ruby riak client?
>>> Stream may well work, but I'm going to have to make a number of changes on the client side to deal with the results.
>>> Sean
>>> On Jul 26, 2013, at 3:53 PM, Brian Roach <roach at basho.com> wrote:
>>>> Sean -
>>>> The timeout isn't via a header, it's a query param -> &timeout=xxxx
>>>> You can also use stream=true to stream the results.
>>>> - Roach
>>>> Sent from my iPhone
>>>> On Jul 26, 2013, at 3:43 PM, Sean McKibben <graphex at graphex.com> wrote:
>>>>> We just upgraded to 1.4 and are having a big problem with some of our larger 2i queries. We have a few key queries that takes longer than 60 seconds (usually about 110 seconds) to execute, but after going to 1.4 we can't seem to get around a 60 second timeout.
>>>>> I've tried:
>>>>> curl -H "X-Riak-Timeout: 260000" "" -i
>>>>> But I always get
>>>>> HTTP/1.1 500 Internal Server Error
>>>>> Vary: Accept-Encoding
>>>>> Server: MochiWeb/1.1 WebMachine/1.10.0 (never breaks eye contact)
>>>>> Date: Fri, 26 Jul 2013 21:41:28 GMT
>>>>> Content-Type: text/html
>>>>> Content-Length: 265
>>>>> Connection: close
>>>>> <html><head><title>500 Internal Server Error</title></head><body><h1>Internal Server Error</h1>The server encountered an error while processing this request:<br><pre>{error,{error,timeout}}</pre><P><HR><ADDRESS>mochiweb+webmachine web server</ADDRESS></body></html>
>>>>> Right at the 60 second mark. What can I set to give my secondary index queries more time??
>>>>> This is causing major problems for us :(
>>>>> Sean
>>>>> _______________________________________________
>>>>> riak-users mailing list
>>>>> riak-users at lists.basho.com
>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>> _______________________________________________
>>> riak-users mailing list
>>> riak-users at lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

More information about the riak-users mailing list