riak-users Digest, Vol 78, Issue 13

Ivaylo Panitchkov ipanitchkov at hibernum.com
Mon Jan 18 09:48:07 EST 2016


>
> Message: 1
> Date: Fri, 15 Jan 2016 18:24:08 +0100
> From: Eugene Shubin <ev.shuba at gmail.com>
> To: riak-users at lists.basho.com
> Subject: Re: Map after reduce
> Message-ID:
>         <CAD6t9YkEP6tGjr5YDHY=
> RMU_oKtkwr52+kDCqhTJCgMDBbdRUA at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Did somebody successfully used trick described here
> <http://cartesianfaith.com/2011/07/27/mapreduce-tips-and-tricks-in-riak/>
> to
> pass reduce results to map phase?
> For me it does not work and looks strange in general.
>
> 2016-01-13 13:55 GMT+01:00 Eugene Shubin <ev.shuba at gmail.com>:
>
> > Is it possible to put a map phase in chain of map-reduce job after reduce
> > phase?
> >
> > Any example, would be highly appreciated.
> >
> > Best regards,
> > Eugene Shubin
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20160115/55a949bf/attachment-0001.html
> >
>
> ------------------------------
>

Hi Eugene,

Passing reduce phase's results to a map phase is mostly useful when you
want to prepare your data for the actual processing.
Here is a piece of my code (node.js) that pulls keys and loads Riak objects
in blocks:

riak.mapreduce
.add(config.sourceBucket)
.reduce({"keep" : false, "language" : "erlang", "module" :
"riak_kv_mapreduce", "function" : "reduce_identity"})
.reduce({"keep" : false, "language" : "javascript", "source" :
"function(value, arg) { return value.slice(0, arg.n); }", "arg" : {"n"
: config.objectsPerBatch, "reduce_phase_batch_size" :
config.objectsPerBatch, "do_prereduce" : true}})
.map({"keep" : true, "language" : "javascript", "source" :
"function(o) { return ('true' !=
o.values[0].metadata['X-Riak-Deleted'])?[{key: o.key, data:
JSON.parse(o.values[0].data)}]:[]; }"})
.run(function(err, objects)
{

 console.log("HEY!");
});








-- 


*Ivaylo Panitchkov*Software developer | Programming
+1 514 504 4297
ipanitchkov at hibernum.com

Facebook <https://www.facebook.com/hibernumcreations/> | Twitter
<https://twitter.com/Hibernum_> | LinkedIn
<https://www.linkedin.com/company/1156110?trk=tyah&trkInfo=clickedVertical%3Acompany%2CclickedEntityId%3A1156110%2Cidx%3A3-1-4%2CtarId%3A1447872522932%2Ctas%3Ahibernum>

<http://www.hibernum.com>

*Ce courriel est confidentiel et peut aussi être protégé par la loi. Si
vous avez reçu ce courriel par erreur, veuillez nous en aviser
immédiatement en y répondant, puis supprimer ce message de votre système.
Veuillez ne pas le copier, l’utiliser pour quelque raison que ce soit ni
divulguer son contenu à quiconque. // **This email is confidential and may
also be legally privileged. If you have received this email in error,
please notify us immediately by reply email and then delete this message
from your system. Please do not copy it or use it for any purpose or
disclose its content.*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20160118/e14b42bd/attachment-0002.html>


More information about the riak-users mailing list