GET vs map reduce

Will Moss wmoss at bu.mp
Fri Aug 19 14:58:32 EDT 2011


It seems like this is a problem that can be solved without too much headache
by the client library. If you're working in an asynchronous environment then
forking off n green threads for each get and collecting the results is no
problem (this is what we do with great results). In a traditionally threaded
environment that could potentially be a bad idea, but you could create a
pool of worker threads that process incoming requests from all your other
processes.

Will


On Fri, Aug 19, 2011 at 9:35 AM, Wilson MacGyver <wmacgyver at gmail.com>wrote:

> this is certainly a tricky problem for us, because there are times we need
> to perform 9 to 100 GETs on per page render basis. some bullt-in
> support would certainly be useful
>
> On Fri, Aug 19, 2011 at 10:12 AM, Jacques <whshub at gmail.com> wrote:
> > This begs the question, is there much efficiency to gain by creating a
> true
> > multi-get?  It seems like a number of people are trying to get the most
> > efficient multi-get possible.
> > If I remember, it was closed as a wontfix a couple years ago.  Did you
> guys
> > at Basho find that it just didn't have that much impact on performance?
> > Thanks,
> > Jacques
> >
> > On Fri, Aug 19, 2011 at 5:45 AM, David Smith <dizzyd at basho.com> wrote:
> >>
> >> As Matt and Jacques noted, M/R is really not intended to be used in
> >> this manner (i.e. for multi-get), particularly if you're interested in
> >> latency. Generally, you will wind up involving more nodes on a M/R
> >> request and shipping the "get" results across those nodes as the
> >> system tries to distribute M/R.
> >>
> >> D.
> >
> >
>
>
>
> --
> Omnem crede diem tibi diluxisse supremum.
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110819/0d325a44/attachment.html>


More information about the riak-users mailing list