Erlang Client Example for using links in map/reduce

Bryan Fink bryan at basho.com
Sat Oct 10 14:07:57 EDT 2009


On Sat, Oct 10, 2009 at 11:12 AM, Paul Jones <pauljones23 at gmail.com> wrote:
> Hi Bryan,
>
> On Sat, Oct 10, 2009 at 2:41 PM, Bryan Fink <bryan at basho.com> wrote:
>>
>> I'll describe how to manage objects you have requested over the HTTP
>> interface, since I expect that's what most people are dealing with.
>
> I'm actually using the Erlang interface, so I poked into the Erlang code a
> bit.

Doh!  I should have paid more attention to your subject line.  Sorry.

>> For now, I'll just give a clue about how you could support links in an
>> entirely different format of data inside Riak (using the raw-Erlang
>> interface instead of the HTTP interface): grep linkfun src/*.erl.  If
>> someone is really interested in such things, and needs a few more
>> pointers, I'll write a second how-to.
>>
>
> So my understanding of this, after looking through the code is that a {link,
> _, _, _} tag simply transforms into a map phase, using whatever link
> function is associated with the bucket.  So if I wanted to the same thing in
> the native Erlang, I take it that I have two options: associate a linkfun
> with the bucket, and use {link,_,_,_}; or simply specify two map phases -
> the first to extract a set of {bucket, key} or {{bucket, key}, data} tuples;
> the second phase then processing the values loaded as a result of these
> phases, returning values ready for a reduce function.
>
> Am I right in this understanding?

Fortuntely, you're good at following hints.  You're right on the money
here.  {link,_,_,_} is, indeed, syntactic sugar for a map phase, and
the linkfun bucket property will allow you to specify the
link-extractor/matcher, such that you can store the links in your data
in whatever format best fits your application.

I think the only other useful tip I can provide at the moment is that
riak_client:reload_all/1 is a useful tool for getting all Riak nodes
to reload the modules you have referenced in your {modfun, Module,
Function} tuples while you're experimenting.

Happy to give more help if needed, though.

-Bryan




More information about the riak-users mailing list