keys=stream

Sean Cribbs sean at basho.com
Tue May 4 09:46:04 EDT 2010


This looks like a bug. I'll add an issue to the github tracker.

Sean Cribbs <sean at basho.com>
Developer Advocate
Basho Technologies, Inc.
http://basho.com/

On May 4, 2010, at 9:29 AM, Adam Hunter wrote:

> Just curb, I couldn't get net/http to work when doing chunked (something about read_body being called twice).
> 
> 
> On May 4, 2010, at 9:28 AM, Sean Cribbs wrote:
> 
>> Are you using curb or net/http?
>> 
>> Sean Cribbs <sean at basho.com>
>> Developer Advocate
>> Basho Technologies, Inc.
>> http://basho.com/
>> 
>> On May 4, 2010, at 9:26 AM, Adam Hunter wrote:
>> 
>>> The problem I was running into was some of the chunks were only part of the json object.  Decoded it would be something like this:
>>> 
>>> chunk 1: {'keys':[1,2,3,4
>>> chunk 2: 5,6,7,8,9]}
>>> 
>>> I was running this on a bucket with about 11k keys.  Using ripple I could only get about 150 keys back this way because all the other chunks were not valid json.  I hacked around this for now by checking to see if the chunk ends in ]}, otherwise I accumulate chunks until I have a full json object.
>>> 
>>> Thanks,
>>> 
>>> Adam
>>> 
>>> 
>>> On May 4, 2010, at 9:22 AM, Sean Cribbs wrote:
>>> 
>>>> This behavior is unchanged since its implementation in 0.8 or 0.9.  To better demonstrate what that statement means, here's the joined output of a streamed keys response:
>>>> 
>>>> {"props":{"name":"foo","n_val":3,"allow_mult":false,"precommit":[],"postcommit":[],"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"old_vclock":86400,"young_vclock":20,"big_vclock":50,"small_vclock":10}}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}{"keys":[]}
>>>> 
>>>> If your client library supports per-chunk processing, each of those {"keys":[]} objects will be an encoded chunk.  The Ruby client and most of the other clients support this.
>>>> 
>>>> Sean Cribbs <sean at basho.com>
>>>> Developer Advocate
>>>> Basho Technologies, Inc.
>>>> http://basho.com/
>>>> 
>>>> On May 3, 2010, at 9:28 PM, Adam Hunter wrote:
>>>> 
>>>>> Hi All,
>>>>> 
>>>>> I apologize if this has been covered already.
>>>>> 
>>>>> In the wiki it says "If keys=stream, the response will be transferred using chunked-encoding, where each chunk is a JSON object" (on http://wiki.basho.com/display/RIAK/REST+API)
>>>>> 
>>>>> This doesn't seem to be the case, sometimes a json object is broken into several chunks.  Is this behavior correct?
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Adam
>>>>> _______________________________________________
>>>>> riak-users mailing list
>>>>> riak-users at lists.basho.com
>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>> 
>>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20100504/19e8de5a/attachment.html>


More information about the riak-users mailing list