Map Reduce Requirements
jeremiah.peschka at gmail.com
Mon Aug 22 15:27:13 EDT 2011
Good questions! I've CC'd the list back in :)
Jeremiah Peschka - Founder, Brent Ozar PLF, LLC
Microsoft SQL Server MVP
On Aug 22, 2011, at 12:18 PM, bill robertson wrote:
> That makes sense.
You could write an initial map phase in Erlang that takes your Protocol Buffers and returns it to the next phase as JSON. As fas as I know, you can combine multiple languages in multiple phases.
I would benchmark this because with a PB -> JSON conversion you still need to convert JSON into an object, but that shouldn't take much at all.
> Additionally, are secondary indexes meta-data? i.e. If I built some secondary indices, these are stored in some form internal to Riak, and therefore available for query regardless of the type of data its associated with. Is this correct?
Secondary indexes are a separate physical structure, or so I gather. (Rusty could be full of lies.) They're stored separately from the initial data and not as metadata in the object headers. So, yes, you can store whatever you want in secondary indexes and query it however you want, provided there's an API that supports what you're doing.
If you're stretching for an analog, you can think of them as similar to secondary indexes in an RDBMS, but that's a stretch for many reasons ;)
> Bill Robertson
> On Mon, Aug 22, 2011 at 2:57 PM, Jeremiah Peschka <jeremiah.peschka at gmail.com> wrote:
> You can MR across whatever kind of data you'd like. JSON is typically used because it's very easy to show people how to query JSON and the structure makes sense to many programmers.
> Jeremiah Peschka - Founder, Brent Ozar PLF, LLC
> Microsoft SQL Server MVP
> On Aug 22, 2011, at 11:51 AM, bill robertson wrote:
> > In order to run a map reduce query v.s. Riak, does the data need to be stored in JSON? If this isn't a requirement, then how would I run a query against data stored in a google protocol buffer format? Is there an example of this somewhere?
> > Thanks!
> > _______________________________________________
> > riak-users mailing list
> > riak-users at lists.basho.com
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
More information about the riak-users