error parsing Link headers in node.js when getting a bucket

Rick Olson technoweenie at gmail.com
Mon May 10 11:56:06 EDT 2010


Ah thanks, I didn't know about the keys=false option.  I eventually
found the key streaming feature an built it into my rial wrapper.  The
weird problems with link headers went away.

On Monday, May 10, 2010, Sean Cribbs <sean at basho.com> wrote:
> Rick,
>
> If you don't want to list the keys when retrieving the bucket properties, pass the keys=false query string.  When you have lots of keys, it's best to use keys=stream to retrieve them, which will prevent Riak from building a huge JSON object in memory. Instead, each chunk in the stream will be a JSON object containing a number of keys.
>
> Sean Cribbs <sean at basho.com>
> Developer Advocate
> Basho Technologies, Inc.
> http://basho.com/
>
> On May 10, 2010, at 3:29 AM, Rick Olson wrote:
>
>> Hey guys, I enjoyed going through the Riak Fast Track, so I wrote a
>> node.js library for it:
>>
>> http://techno-weenie.net/2010/5/10/rhodes-riak-on-node-js
>>
>> I ran into one problem though: node.js choked on the Link headers when
>> getting a bucket's properties.  I realize this is a node.js bug, but I
>> found it kind of odd:
>>
>>  $ curl http://127.0.0.1:8091/riak/goog -i
>>  HTTP/1.1 200 OK
>>  Vary: Accept-Encoding
>>  Server: MochiWeb/1.1 WebMachine/1.6 (eat around the stinger)
>>  Link: </riak/goog/2006-05-22>; riaktag="contained",
>> </riak/goog/2006-05-04>; riaktag="contained", ETC
>>
>> If there are 1400 links, node.js has problems.  If there are several
>> thousand, even curl starts having problems:
>>
>>  curl: (27) Avoided giant realloc for header (max is 102400)!
>>
>> So, if I grab the contents of a bucket to get the list of keys, it
>> also sends a giant Link header pointing to each key?  The Link header
>> ends up being much larger than the response body.  Is it really
>> necessary to point to each resource, when the body contains an array
>> of keys?
>>
>> At any rate, here's how the map reduce API looks while implementing
>> the sample functions from the fast track:
>>
>> http://github.com/technoweenie/rhodes/blob/master/examples/fast-track/mapred.js#L6-19
>>
>> --
>> Rick Olson
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>

-- 
Rick Olson




More information about the riak-users mailing list