MapReduce questions

Mark Phillips mark at basho.com
Mon Oct 22 12:18:29 EDT 2012


Hi C,

On Fri, Oct 19, 2012 at 10:04 AM, Callixte Cauchois
<ccauchois at virtuoz.com> wrote:
> Hey Mark,
>
> really sorry if I sounded aggressive or whatever. English is not my primary
> language and sometimes I do not sound as I wanted... I just wanted to
> acknowledge that no answer was kind of an answer to my questions.

No worries. And sorry about my tone if it was too strong. Responding
to emails before any coffee is not recommended. :)

> And yes, I will share how I would like M/R to behave for future reference,
> even if it is not going to be implemented short or mid term. Unfortunately,
> I do not speak Erlang, so it would be hard for me to contribute, even if I
> would love to. Maybe, I'd feel extra adventurous and give it a try though.
> Thank you for your response, and I also wanted to say that I love the new
> documentation site, compared to how it can be a struggle to understand how
> some competitors'product can be installed, it really kicks ass.
>

Erlang can be a bit daunting to start, but once you dive in you'll
probably be hooked. (Don't say I didn't warn you.) If you're looking
for some Riak-related ways to test it out, there are a few contributed
erlang functions [1] that might be worth examining. Also, writing
riaknostic diagnostics [2] is an easy way to start learning. We're
also happy to help on this list where possible.

The docs have needed a lot of love for some time. There is a lot more
good stuff coming for docs, resources, etc. Glad you like them. Make
sure to tell us where they need some love.

Best,

Mark

[1] http://contrib.basho.com/map-reduce-functions.html
[2] http://riaknostic.basho.com

> C.
>
>
> On Fri, Oct 19, 2012 at 9:52 AM, Mark Phillips <mark at basho.com> wrote:
>>
>> On Fri, Oct 19, 2012 at 8:48 AM, Callixte Cauchois
>> <ccauchois at virtuoz.com> wrote:
>> > So... no answers.
>> > I guess there are no smart minds at Basho working on M/R currently. Too
>> > bad,
>> > but I guess a company has to choose its priority.
>>
>> A lovely "good morning" to you, too.
>>
>> >
>> as part of my evaluation of Riak, I am looking at the M/R capabilities
>> and I have several questions:
>> 1/ the doc states that " Riak MapReduce is intended for batch
>> processing, not real-time querying." But as of now, you always get the
>> results and cannot automatically store them in a bucket like you would
>> do with MongoDB for example. Is taht something that is on the roadmap?
>> >
>>
>> This isn't something that's on the roadmap. That doesn't mean we won't
>> do it, but no one is working on code that does this at the moment.
>> That said, I'm told by one of our smart, busy minds here that works on
>> m/r that it would not be the most complicated addition. It may well
>> end up being part of future work but, again, there's no timeline on
>> it.  In the mean time, any use case details you could share (short of
>> "like you would do with MongoDB") that would help inform the
>> development would be stellar.
>>
>> If you were feeling adventurous and wanted to write some erlang code,
>> this behavior could be written as part of map or reduce job.
>>
>> >
>> 2/ Basho blog has two articles from last year on Hadoop on top of
>> Riak. Is this project still live? It is on github, so I can always dig
>> it up and work on it, but I am wondering if there is something like
>> Yokozuna for Hadoop integration.
>> >
>>
>> The work on this specific code has stalled. That said, there are some
>> people using Riak in production alongside hadoop, and we have plans to
>> build deeper integration between the two at some point.
>>
>> As an aside, if you're looking for lower latency responses on
>> implementation questions, you might want to consider hanging out in
>> #riak on freenode.
>>
>> Mark
>> twitter.com/pharkmillups
>>
>> >
>> >
>> > On Tue, Oct 16, 2012 at 11:03 AM, Callixte Cauchois
>> > <ccauchois at virtuoz.com>
>> > wrote:
>> >>
>> >> Hi there,
>> >>
>> >> as part of my evaluation of Riak, I am looking at the M/R capabilities
>> >> and
>> >> I have several questions:
>> >> 1/ the doc states that " Riak MapReduce is intended for batch
>> >> processing,
>> >> not real-time querying." But as of now, you always get the results and
>> >> cannot automatically store them in a bucket like you would do with
>> >> MongoDB
>> >> for example. Is taht something that is on the roadmap?
>> >> 2/ Basho blog has two articles from last year on Hadoop on top of Riak.
>> >> Is
>> >> this project still live? It is on github, so I can always dig it up and
>> >> work
>> >> on it, but I am wondering if there is something like Yokozuna for
>> >> Hadoop
>> >> integration.
>> >>
>> >> Thanks.
>> >> C.
>> >
>> >
>> >
>> > _______________________________________________
>> > riak-users mailing list
>> > riak-users at lists.basho.com
>> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> >
>
>




More information about the riak-users mailing list