slow mapred_search key lookups for single terms

Bryan Fink bryan at basho.com
Fri Apr 6 16:03:34 EDT 2012


On Fri, Apr 6, 2012 at 11:00 AM, Michael Radford <mrad at blorf.com> wrote:
> Calling riak_kv_mrc_pipe directly from the console is just as slow as
> using the pb interface.

Thank you for confirming that.  Much appreciated.

> it seemed to be encoding one PB message per search result and
> spending quite a bit of time doing it
> …snip…
> So if there's an opportunity
> for batching the search results, that seems like it would be a big
> improvement as well.
>
> (It looks like that is currently up to whatever process is sending the
> results in the first place. But maybe it would be a good idea for the
> pb socket to accumulate M/R results until it reaches some threshold of
> bytes to send...calling term_to_binary to estimate is pretty fast.)

Indeed, you're correct here as well.  I think there was a TODO about
just this at some point, but it seems to have been lost in an
issue-tracker shuffle.  I've refiled it at
https://github.com/basho/riak_kv/issues/312

Thanks,
Bryan




More information about the riak-users mailing list