Storing objects during MapReduce phase.

ttt ttt at ist.utl.pt
Tue Feb 12 09:32:54 EST 2013


Hi everyone!


Is it possible to store data into Riak during a Map or Reduce phase?

This is my scenario:
- I have a simple MapReduce job, with 1 map followed with 1 reduce.
- I am using the Ruby library and JavaScript to code the phase functions.
- In the map phase, some properties of the input objects are recalculated
and then passed on to the reduce phase.
- I am keeping the results from the map phase, to update the input objects
as soon as they come back via streaming.
- The MapReduce job is used in each iteration of an implementation of the
K-means clustering algorithm.

Once the map phase runs on the nodes where the objects are stored, it would
be preferable to have them being updated by the JavaScript function that
runs on the map phase, to avoid the data objects being transfered back and
forth.

Would it be a good idea to install the JavaScript client library on the
server and make it available to the MapReduce phase functions? Would it
work?


Thanks for your time.
Best regards.
Tiago



--
View this message in context: http://riak-users.197444.n3.nabble.com/Storing-objects-during-MapReduce-phase-tp4026852.html
Sent from the Riak Users mailing list archive at Nabble.com.




More information about the riak-users mailing list