Map Reduce Requirements

bill robertson billrobertson42 at gmail.com
Mon Aug 22 23:50:22 EDT 2011


On Mon, Aug 22, 2011 at 3:27 PM, Jeremiah Peschka <
jeremiah.peschka at gmail.com> wrote:

> I don't think Erlang can talk to JavaScript inside a single
> phase/function/pile of source code. I could be wrong, but it seems to me
> that marshaling data across the JavaScript/Erlang boundary would be hella
> expensive and cause a lot of problems and, as such, probably doesn't exist.
>

It certainly seems sub-optimal.

I wonder if it would be feasible to deploy an erlang web-service in the riak
node's webmachine instance that could translate meta-data into Erlang funs
and drive the map reduce operation that way. I'm not sure if I could get
around having specific knowledge of the protobuf structures baked into that
code, but I don't think it matters in this case.

I also wonder how much 1.0 will change this picture.

> Additionally, are secondary indexes meta-data?  i.e. If I built some
> secondary indices, these are stored in some form internal to Riak, and
> therefore available for query regardless of the type of data its associated
> with. Is this correct?
>
> Secondary indexes are a separate physical structure, or so I gather. (Rusty
> could be full of lies.) They're stored separately from the initial data and
> not as metadata in the object headers. So, yes, you can store whatever you
> want in secondary indexes and query it however you want, provided there's an
> API that supports what you're doing.
>

Would secondary indexes eliminate the need for key-filtering? Logically, it
would seem that you could do with indexes, but do they have similar
performance characteristics?  (i.e. does one suck more than the other?)

Thanks again,
Bill Robertson
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110822/1bcbc860/attachment.html>


More information about the riak-users mailing list