map/reduce in none-erlang?

Andy Gross andy at basho.com
Tue Dec 15 08:51:52 EST 2009


Hi Amos,


1. do I HAVE to use Erlang to implement this or can I link code from
> other languages?
>

Currently, map/reduce code has to be written in Erlang.  However, in early
2009 we'll be releasing Javascript integration, which will allow you to
write custom map-reduce jobs in JS.  We do have some common map/reduce
functions pre-packaged with Riak - see
http://riak.basho.com/edoc/riak_mapreduce.html for details.


> 2. where does this code have to be installed? Is it part of the Riak
> client like in the example?
>

Currently, any Erlang modules referenced by a map/reduce job must be
installed on all machines in the cluster.   The riak client module has
convenience functions for reloading new versions of these modules.


> 3. Where does it get executed? Is it some Erlang magic that gets it
> distribute execution on multiple computers?
>

Map functions are executed "where the data lives", using Riak's consistent
hashing functions.


> 4. Does it make sense to use Riak only as a distributed key/value data
> store for very large data sets, without taking advantage of map/reduce
> (because our core code base is in C++)?
>

Absolutely - many users and customers use Riak for key-value access.


> 5. Am I correct to understand that map/reduce cannot be implemented on
> top of the HTTP interface Riak provides (or at least it'll miss the
> performance boost if implemented that way)?
>

Currently that is the case.   When we release Javascript integration, you'll
be able to upload arbitrary map/reduce functions over HTTP.

 Link-walking (which is just syntactic sugar over a specialized case of
map/reduce) can be done over the HTTP interface.

See :  http://riak.basho.com/edoc/jaywalker_resource.html for the HTTP
linkwalking syntax.


>
> Any pointers to more elaborate map/reduce examples or documents which
> can help me learn more, possibly using other languages, would be
> appreciated. I think I pretty much covered the web site for
> introductory level docs.
>

This screencast with Bryan Fink has some live examples of link walking:
http://www.vimeo.com/7196242

Hope this helps,

Andy Gross
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20091215/f448cb24/attachment-0002.html>


More information about the riak-users mailing list