Fwd: Map Reduce Requirements
billrobertson42 at gmail.com
Mon Aug 22 15:19:08 EDT 2011
Forgot to reply-all...
---------- Forwarded message ----------
From: bill robertson <billrobertson42 at gmail.com>
Date: Mon, Aug 22, 2011 at 3:18 PM
Subject: Re: Map Reduce Requirements
To: Jeremiah Peschka <jeremiah.peschka at gmail.com>
That makes sense.
Assume that I can write an Erlang function called F that will translate the
raw GPB bytes into the appropriate JSON for use by Q1. How would I hook F
into the processing of Q1?
phase at which point I could call my translation function and operate on the
JSON, and possibly pass on a structure containing the JSON and the GPB to
the next phase.
Does that make sense? Is it possible to invoke arbitrary Erlang functions
Additionally, are secondary indexes meta-data? i.e. If I built some
secondary indices, these are stored in some form internal to Riak, and
therefore available for query regardless of the type of data its associated
with. Is this correct?
On Mon, Aug 22, 2011 at 2:57 PM, Jeremiah Peschka <
jeremiah.peschka at gmail.com> wrote:
> You can MR across whatever kind of data you'd like. JSON is typically used
> because it's very easy to show people how to query JSON and the structure
> makes sense to many programmers.
> To MR across anything else, you'll want a library that will translate your
> protocol buffers encoded data into objects that can be parsed in either
> Serialization/Deserialization function to translate between data at rest
> (protobufs) to data that the MR program can understand.
> Since there are protocol buffer libraries for many languages, this should
> it shouldn't be much more difficult than Riak.mapValuesJson - provided that
> you can find some easy magic to translate objects for you ;)
> Jeremiah Peschka - Founder, Brent Ozar PLF, LLC
> Microsoft SQL Server MVP
> On Aug 22, 2011, at 11:51 AM, bill robertson wrote:
> > In order to run a map reduce query v.s. Riak, does the data need to be
> stored in JSON? If this isn't a requirement, then how would I run a query
> against data stored in a google protocol buffer format? Is there an example
> of this somewhere?
> > Thanks!
> > _______________________________________________
> > riak-users mailing list
> > riak-users at lists.basho.com
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the riak-users