Loading links after a map phase in a map reduce query

Sean Cribbs sean at basho.com
Thu Oct 7 09:08:58 EDT 2010


Jan,

It is possible to do a link phase after a map phase, if your map phase outputs lists of [bucket,key] pairs.  However, the semantics of the link phase will be to load those specified objects and follow *their* links.  If you just want to extract the links from the body of your first object and follow them, do so in the preceding map phase, and then your next map phase will be over the target of the links.

Regarding the size of the Link header, if it's possible to modify your client, make it emit multiple Link headers that are less than 8KB in length (which has the same semantics as a single Link header with items separated by commas).  The Ruby client does this:

http://github.com/seancribbs/ripple/blob/master/riak-client/lib/riak/util/headers.rb#L22

Sean Cribbs <sean at basho.com>
Developer Advocate
Basho Technologies, Inc.
http://basho.com/

On Oct 7, 2010, at 8:50 AM, Jan Metzner wrote:

> Hi folks,
> 
> i am experimenting with riak and have a problem with too many links in a riak object.
> If i store the object the server send 400 Bad Request. This is caused from the link header which is too long.
> As workaround i thought about storing the links inside the riak object and fetch them in a map reduce query.
> Is it possible to do a link phase after a map phase in riak and if so has somebody an example for this?
> 
> Thanks and regards
> Jan
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20101007/a39d04e1/attachment.html>


More information about the riak-users mailing list