error parsing Link headers in node.js when getting a bucket

Sean Cribbs sean at basho.com
Mon May 10 08:02:21 EDT 2010


Rick,

If you don't want to list the keys when retrieving the bucket properties, pass the keys=false query string.  When you have lots of keys, it's best to use keys=stream to retrieve them, which will prevent Riak from building a huge JSON object in memory. Instead, each chunk in the stream will be a JSON object containing a number of keys.

Sean Cribbs <sean at basho.com>
Developer Advocate
Basho Technologies, Inc.
http://basho.com/

On May 10, 2010, at 3:29 AM, Rick Olson wrote:

> Hey guys, I enjoyed going through the Riak Fast Track, so I wrote a
> node.js library for it:
> 
> http://techno-weenie.net/2010/5/10/rhodes-riak-on-node-js
> 
> I ran into one problem though: node.js choked on the Link headers when
> getting a bucket's properties.  I realize this is a node.js bug, but I
> found it kind of odd:
> 
>  $ curl http://127.0.0.1:8091/riak/goog -i
>  HTTP/1.1 200 OK
>  Vary: Accept-Encoding
>  Server: MochiWeb/1.1 WebMachine/1.6 (eat around the stinger)
>  Link: </riak/goog/2006-05-22>; riaktag="contained",
> </riak/goog/2006-05-04>; riaktag="contained", ETC
> 
> If there are 1400 links, node.js has problems.  If there are several
> thousand, even curl starts having problems:
> 
>  curl: (27) Avoided giant realloc for header (max is 102400)!
> 
> So, if I grab the contents of a bucket to get the list of keys, it
> also sends a giant Link header pointing to each key?  The Link header
> ends up being much larger than the response body.  Is it really
> necessary to point to each resource, when the body contains an array
> of keys?
> 
> At any rate, here's how the map reduce API looks while implementing
> the sample functions from the fast track:
> 
> http://github.com/technoweenie/rhodes/blob/master/examples/fast-track/mapred.js#L6-19
> 
> -- 
> Rick Olson
> 
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com





More information about the riak-users mailing list