Map/Reduce In pure erlang

Brian Rowe rowe at muxspace.com
Fri Jul 29 12:27:25 EDT 2011


Glad to be of help. For yet another shameless plug, I'm actually
tinkering with my branch of Rekon to make interacting with erlang
modules (modify code paths, create, compile, load modules on the fly)
more convenient. Obviously there are security considerations, but for
now I think the M/R interface will benefit from Lewis & Clark-style
exploration.

Brian


On Fri, Jul 29, 2011 at 12:17 PM, Jeremiah Peschka
<jeremiah.peschka at gmail.com> wrote:
> WOOHOO! My wrong answer brought out the right one. Thanks for the creative solution, Brian!
> ---
> Jeremiah Peschka
> Founder, Brent Ozar PLF, LLC
>
> On Jul 29, 2011, at 9:09 AM, Brian Rowe wrote:
>
>> This isn't completely accurate. There's nothing to stop you from
>> writing a module that reads in a key, compiles the payload as code,
>> and calls a function within the newly compiled module. Similarly, you
>> can set up the nodes to point to an external library path and run the
>> modules from there.
>>
>> As far as running map/reduce jobs from erlang, yes, there is an erlang
>> API for that using the riak erlang client. You can see from the source
>> that all of these operations are available:
>>
>> -export([start_link/2, start_link/3,
>>         start/2, start/3,
>>         stop/1,
>>  ... snip ...
>>         mapred/3, mapred/4, mapred/5,
>>         mapred_stream/4, mapred_stream/5, mapred_stream/6,
>>         mapred_bucket/3, mapred_bucket/4, mapred_bucket/5,
>>         mapred_bucket_stream/5, mapred_bucket_stream/6,
>>
>> https://github.com/basho/riak-erlang-client/blob/master/src/riakc_pb_socket.erl
>>
>>
>> On Fri, Jul 29, 2011 at 11:12 AM, Jeremiah Peschka
>> <jeremiah.peschka at gmail.com> wrote:
>>> As I remember it, if you want to do Map/Reduce with erlang, you can only use functions that have been compiled and uploaded to the Riak cluster. That is: there's no way to write your erlang M/R jobs on the fly like you can do with JavaScript
>>> ---
>>> Jeremiah Peschka
>>> Founder, Brent Ozar PLF, LLC
>>>
>>> On Jul 29, 2011, at 8:10 AM, Lyes Amazouz wrote:
>>>
>>>> Hi!
>>>>
>>>> I want to know if there is a way to do a map/Reduce query using pure erlang library without (even for the map/reduce query functions) without using the Javascript or passing by the http interface . Does riak offers something like that?
>>>>
>>>> Thank you!
>>>>
>>>> --
>>>> ================
>>>> Lyes
>>>> ================
>>>>
>>>> _______________________________________________
>>>> riak-users mailing list
>>>> riak-users at lists.basho.com
>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> riak-users at lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>
>




More information about the riak-users mailing list