CorrugatedIron Link Walking
rkevinburton at charter.net
Thu Feb 7 08:50:36 EST 2013
First off thank you. I dont know why the learning curve is so steep for me.
I am just looking at the example on CIs web site. I have seen the warnings
about sending a whole bucket as input.
Rather than specifying what links you are interested in what about all the
links associated with the objects on the input. Is something like .Link(l =>
l) valid? Or is there a shorthand version?
The reduce is still pretty much a black box to me. On the Basho site there
are a number of examples of link walking where no reduce is used. Also it is
unclear to me what the functions passed as arguments do in the
ReduceErlang call. What is ModFun? What does riak_kv_mapreduce do? And
what about reduce_set_union? The Keep attribute is documented on the
Basho site so I think I understand it.
I am looking at
g/ It is almost like the links that are to be walked are keys.
The first example they have is:
curl -v http://127.0.0.1:8091/riak/people/timoreilly/people,friend,1
Then there is the explanation:
You'll notice that at the end of that request we've tacked on
/people,friend,1 That is the link specification. It's composed of three
§ Bucket - a bucket name to limit the links to (in the above request it's
* Tag - the riaktag to limit the links ('friend' is the tag in the
§ Keep - 0 or 1, whether to return results from this step or phase
None of this talks about the use of a special Erlang reduce function.
Then on http://docs.basho.com/riak/latest/tutorials/querying/MapReduce. The
example give doesnt have a reduce function. Just a map. And the sample uses
The documentation mentions a list of phases that go along with the input.
For these simple examples I dont see any phases listed. I guess I still
dont understand phases.
MapReduce queries in Riak have two components: a list of inputs and a list
of steps, or phases.
Each element of the input list is a bucket-key pair. This bucket-key pair
may also be annotated with key-data, which will be passed as an argument
to a map function, when evaluated on the object stored under that bucket-key
I am having a hard time correlating the documentation with implementation.
Any help would be greatly appreciated.
From: OJ Reeves [mailto:oj at buffered.io]
Sent: Thursday, February 07, 2013 3:25 AM
To: Kevin Burton
Cc: riak-users at lists.basho.com
Subject: Re: CorrugatedIron Link Walking
A few points:
* It's recommended that you avoid passing in a whole bucket as an
argument to a map/reduce query as this results in keys for the bucket being
listed, which as you know is an expensive operation.
* The .Link() function is a function that says "please extract links
from the objects specified by the .Inputs() function".
* The lambda function that is passed to the .Link() function in the
example you listed says "please look for links that are called "friend" that
point at the "people" bucket.
* .ReduceErlang() says "perform a reduce phase on the results from the
previous phase using an Erlang module". The inputs to this phase are the
outputs from the link thanks to the use of the .Link() function.
* When using .ReduceErlang() you use the lambda to specify which
module and function you want called on the inputs. The example below says
"filter out duplicates using the reduce_set_union function that comes with
Riak's riak_kv_mapreduce module".
I hope that clears things up a bit. I would recommend reading Basho's
documentation on map/reduces phases to help you get your head around what
they are and how they work.
Best of luck
On Thu, Feb 7, 2013 at 4:46 PM, Kevin Burton <rkevinburton at charter.net>
I am trying out link walking with CorrugatedIron. I see in the documentation
var query1 = new RiakMapReduceQuery()
.Link(l => l.Tag("friend").Bucket("people"))
.ReduceErlang(r => r.ModFun("riak_kv_mapreduce",
I am unclear what the function of the last ReduceErlang method is?
Also I want all links of a certain tag. Isnt the bucket and key specified
in the link? What is the purpose of the Bucket?
Finally this is probably my ignorance on Map/Reduce but I dont understand
the different phase results.
riak-users mailing list
riak-users at lists.basho.com
+61 431 952 586
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the riak-users