2i timeouts in 1.4

Sean McKibben graphex at graphex.com
Sat Jul 27 01:11:53 EDT 2013


So when I try to use pagination, it doesn't seem to be picking up my continuation. I'm having trouble parsing the json I get back using stream=true (and there is still a timeout) so I went to just using pagination. Perhaps I'm doing it wrong, (likely, it has been a long day) but riak seems to be ignoring my continuation:

(pardon the sanitization)
curl 'http://127.0.0.1:8098/buckets/mybucket/index/test_bin/myval?max_results=5'
{"keys":["1","2","3","4","5"],"continuation":"g20AAABAMDAwMDE1ZWVjMmNiZjY3Y2Y4YmU3ZTVkMWNiZTVjM2ZkYjg2YWU0MGIwNzNjMTE3NDYyZjEzMTNlMDQ5YmI2ZQ=="}

curl 'http://127.0.0.1:8098/buckets/mybucket/index/test_bin/myval?max_results=5&continuation=g20AAABAMDAwMDE1ZWVjMmNiZjY3Y2Y4YmU3ZTVkMWNiZTVjM2ZkYjg2YWU0MGIwNzNjMTE3NDYyZjEzMTNlMDQ5YmI2ZQ=='
{"keys":["1","2","3","4","5"],"continuation":"g20AAABAMDAwMDE1ZWVjMmNiZjY3Y2Y4YmU3ZTVkMWNiZTVjM2ZkYjg2YWU0MGIwNzNjMTE3NDYyZjEzMTNlMDQ5YmI2ZQ=="}

The same keys and continuation value are returned regardless of whether my request contains a continuation value. I've tried swapping the order of max_results and continuation without any luck. I also made sure that my continuation value was url encoded. Hopefully I'm not missing something obvious here. Well, come to think of it, hopefully I am missing something obvious!

Sean

On Jul 26, 2013, at 6:43 PM, Russell Brown <russell.brown at mac.com> wrote:

> For a work around you could use streaming and pagination.
> 
> Request smaller pages of data (i.e. sub 60 seconds worth) and use streaming to get the results to your client sooner.
> 
> In HTTP this would look like
> 
> http://127.0.0.1:8098/buckets/mybucket/index/test_bin/myval?max_results=10000&stream=true
> 
> your results will include a continuation like
> 
>    {"continuation":"g2gCYgAAFXttAAAABDU0OTk="}
> 
> and you can use that to get the next N results. Breaking your query up that way should duck the timeout.
> 
> Furthermore, adding &stream=true will mean the first results is received very rapidly.
> 
> I don't think the Ruby client is up to date for the new 2i features, but you could monkeypatch as before.
> 
> Cheers
> 
> Russell
> 
> On 26 Jul 2013, at 19:00, Sean McKibben <graphex at graphex.com> wrote:
> 
>> Thank you for looking in to this. This is a major problem for our production cluster, and we're in a bit of a bind right now trying to figure out a workaround in the interim. It sounds like maybe a mapreduce might handle the timeout properly, so hopefully we can do that in the meantime.
>> If there is any way we can have a hotfix ASAP though, that would be preferable. Certainly would not be a problem for us to edit a value in the config file (and given the lack of support in the ruby client for the timeout setting, the ability to edit the global default would be preferred).
>> In the ruby client i had to monkeypatch it like this to even submit the timeout value, which is not ideal:
>> 
>> module Riak
>> class Client
>>   class HTTPBackend
>>     def get_index(bucket, index, query)
>>       bucket = bucket.name if Bucket === bucket
>>       path = case query
>>              when Range
>>                raise ArgumentError, t('invalid_index_query', :value => query.inspect) unless String === query.begin || Integer === query.end
>>                index_range_path(bucket, index, query.begin, query.end)
>>              when String, Integer
>>                index_eq_path(bucket, index, query, 'timeout' => '260000')
>>              else
>>                raise ArgumentError, t('invalid_index_query', :value => query.inspect)
>>              end
>>       response = get(200, path)
>>       JSON.parse(response[:body])['keys']
>>     end
>>   end
>> end
>> end
>> 
>> Thanks for the update,
>> Sean
>> 
>> 
>> 
>> On Jul 26, 2013, at 4:49 PM, Russell Brown <russell.brown at mac.com> wrote:
>> 
>>> Hi Sean,
>>> I'm very sorry to say that you've found a featurebug.
>>> 
>>> There was a fix put in here https://github.com/basho/riak_core/pull/332
>>> 
>>> But that means that the default timeout of 60 seconds is now honoured. In the past it was not.
>>> 
>>> As far as I can see the 2i endpoint never accepted a timeout argument, and it still does not.
>>> 
>>> The fix would be to add the timeout to the 2i API endpoints, and I'll do that straight away.
>>> 
>>> In the meantime, I wonder if streaming the results would help, or if you'd still hit the overall timeout?
>>> 
>>> Very sorry that you've run into this. Let me know if streaming helps, I've raised an issue here[1] if you want to track this bug
>>> 
>>> Cheers
>>> 
>>> Russell
>>> 
>>> [1] https://github.com/basho/riak_kv/issues/610
>>> 
>>> 
>>> On 26 Jul 2013, at 17:59, Sean McKibben <graphex at graphex.com> wrote:
>>> 
>>>> I should have mentioned that I also tried:
>>>> curl -H "X-Riak-Timeout: 260000" "http://127.0.0.1:8098/buckets/mybucket/index/test_bin/myval?timeout=260000" -i
>>>> but still receive the 500 error below exactly at the 60 second mark. Is this a bug?
>>>> 
>>>> Secondary to getting this working at all, is this documented anywhere? and any way to set this timeout using the ruby riak client?
>>>> 
>>>> Stream may well work, but I'm going to have to make a number of changes on the client side to deal with the results.
>>>> 
>>>> Sean
>>>> 
>>>> On Jul 26, 2013, at 3:53 PM, Brian Roach <roach at basho.com> wrote:
>>>> 
>>>>> Sean -
>>>>> 
>>>>> The timeout isn't via a header, it's a query param -> &timeout=xxxx
>>>>> 
>>>>> You can also use stream=true to stream the results.
>>>>> 
>>>>> - Roach
>>>>> 
>>>>> Sent from my iPhone
>>>>> 
>>>>> On Jul 26, 2013, at 3:43 PM, Sean McKibben <graphex at graphex.com> wrote:
>>>>> 
>>>>>> We just upgraded to 1.4 and are having a big problem with some of our larger 2i queries. We have a few key queries that takes longer than 60 seconds (usually about 110 seconds) to execute, but after going to 1.4 we can't seem to get around a 60 second timeout.
>>>>>> 
>>>>>> I've tried:
>>>>>> curl -H "X-Riak-Timeout: 260000" "http://127.0.0.1:8098/buckets/mybucket/index/test_bin/myval?x-riak-timeout=260000" -i
>>>>>> 
>>>>>> But I always get
>>>>>> HTTP/1.1 500 Internal Server Error
>>>>>> Vary: Accept-Encoding
>>>>>> Server: MochiWeb/1.1 WebMachine/1.10.0 (never breaks eye contact)
>>>>>> Date: Fri, 26 Jul 2013 21:41:28 GMT
>>>>>> Content-Type: text/html
>>>>>> Content-Length: 265
>>>>>> Connection: close
>>>>>> 
>>>>>> <html><head><title>500 Internal Server Error</title></head><body><h1>Internal Server Error</h1>The server encountered an error while processing this request:<br><pre>{error,{error,timeout}}</pre><P><HR><ADDRESS>mochiweb+webmachine web server</ADDRESS></body></html>
>>>>>> 
>>>>>> Right at the 60 second mark. What can I set to give my secondary index queries more time??
>>>>>> 
>>>>>> This is causing major problems for us :(
>>>>>> 
>>>>>> Sean
>>>>>> _______________________________________________
>>>>>> riak-users mailing list
>>>>>> riak-users at lists.basho.com
>>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>> 
>>>> _______________________________________________
>>>> riak-users mailing list
>>>> riak-users at lists.basho.com
>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>> 
>> 
> 
> 
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com





More information about the riak-users mailing list