CorrugatedIron Link Walking

Kevin Burton rkevinburton at charter.net
Thu Feb 7 08:50:36 EST 2013


First off thank you. I don’t know why the learning curve is so steep for me.

 

I am just looking at the example on CI’s web site. I have seen the warnings
about sending a whole bucket as input.

 

Rather than  specifying what links you are interested in what about all the
links associated with the objects on the input. Is something like .Link(l =>
l) valid? Or is there a shorthand version?

 

The reduce is still pretty much a black box to me. On the Basho site there
are a number of examples of link walking where no reduce is used. Also it is
unclear to me what the functions passed as arguments do in the
‘ReduceErlang’ call. What is ‘ModFun’? What does ‘riak_kv_mapreduce’ do? And
what about ‘reduce_set_union’? The ‘Keep’ attribute is documented on the
Basho site so I think I understand it.

 

I am looking at
http://docs.basho.com/riak/latest/tutorials/fast-track/Links-and-Link-Walkin
g/ It is almost like the links that are to be walked are keys.

 

The first example they have is:

 

curl -v http://127.0.0.1:8091/riak/people/timoreilly/people,friend,1

 

 

Then there is the explanation:

 

You'll notice that at the end of that request we've tacked on
“/people,friend,1” That is the link specification. It's composed of three
parts:

§  Bucket - a bucket name to limit the links to (in the above request it's
'people')

*	Tag - the “riaktag” to limit the links ('friend' is the tag in the
above request)

§  Keep - 0 or 1, whether to return results from this step or phase

None of this talks about the use of a special Erlang reduce function.

 

Then on http://docs.basho.com/riak/latest/tutorials/querying/MapReduce. The
example give doesn’t have a reduce function. Just a map. And the sample uses
JavaScript instead of Erlang. Why choose one over the other?

 

The documentation mentions a list of phases that go along with the input.
For these simple examples I don’t see any phases listed. I guess I still
don’t understand phases.

 

http://docs.basho.com/riak/latest/references/appendices/MapReduce-Implementa
tion/ specifically:

 

MapReduce queries in Riak have two components: a list of inputs and a list
of “steps”, or “phases”.

Each element of the input list is a bucket-key pair. This bucket-key pair
may also be annotated with “key-data”, which will be passed as an argument
to a map function, when evaluated on the object stored under that bucket-key
pair.

I am having a hard time correlating the documentation with implementation.
Any help would be greatly appreciated.

 

 

 

From: OJ Reeves [mailto:oj at buffered.io] 
Sent: Thursday, February 07, 2013 3:25 AM
To: Kevin Burton
Cc: riak-users at lists.basho.com
Subject: Re: CorrugatedIron Link Walking

 

Kevin,

 

A few points:

*	It's recommended that you avoid passing in a whole bucket as an
argument to a map/reduce query as this results in keys for the bucket being
listed, which as you know is an expensive operation.
*	The .Link() function is a function that says "please extract links
from the objects specified by the .Inputs() function".
*	The lambda function that is passed to the .Link() function in the
example you listed says "please look for links that are called "friend" that
point at the "people" bucket.
*	.ReduceErlang() says "perform a reduce phase on the results from the
previous phase using an Erlang module". The inputs to this phase are the
outputs from the link thanks to the use of the .Link() function.
*	When using .ReduceErlang() you use the lambda to specify which
module and function you want called on the inputs. The example below says
"filter out duplicates using the reduce_set_union function that comes with
Riak's riak_kv_mapreduce module".

I hope that clears things up a bit. I would recommend reading Basho's
documentation on map/reduces phases to help you get your head around what
they are and how they work.

 

Best of luck

OJ

 

On Thu, Feb 7, 2013 at 4:46 PM, Kevin Burton <rkevinburton at charter.net>
wrote:

I am trying out link walking with CorrugatedIron. I see in the documentation
(http://corrugatediron.org/documentation/MapReduce.html):

 

var query1 = new RiakMapReduceQuery()

                 .Inputs("people")

                 .Link(l => l.Tag("friend").Bucket("people"))

                 .ReduceErlang(r => r.ModFun("riak_kv_mapreduce",
"reduce_set_union").Keep(true));

 

I am unclear what the function of the last ‘ReduceErlang’ method is?

 

Also I want all links of a certain tag. Isn’t the bucket and key specified
in the link? What is the purpose of the ‘Bucket’?

 

Finally this is probably my ignorance on Map/Reduce but I don’t understand
the different phase results.

 

Thank you.

 


_______________________________________________
riak-users mailing list
riak-users at lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com





 

-- 

OJ Reeves

+61 431 952 586
http://buffered.io/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130207/0dffab64/attachment.html>


More information about the riak-users mailing list