riak-ql

Antonio Rohman Fernandez rohman at mahalostudio.com
Tue Jul 26 23:21:39 EDT 2011


  

"The problem I see with riak-ql and Antonio's thing is that they're
invariably going to be slow.
Javascript Map/Reduce over an entire bucket
is just not suitable for inline requests."

Yes, of course, i also think
that the MapReduce phases should be done in the background with some
cron jobs or other methods... you don't want to execute this kind of
queries on your UI web app, but at least the development is done in case
you need to do so ( of course, using your head on how you distribute
data on the buckets )... is for example with facebook's "News feed"...
if we call that MapReduce query everytime the user click on the "Home"
tab, it will be terrible expensive, so a process in the background
generating the "News feed" for you and updating it every 5min ( for
example ) will be more ideal... but still, is good to have MapReduce
options OnDemand in case you want to grab some special data, etc... even
the transaction will be a bit costly.

Also it depends on how you store
your data... if you just have a "Messages" bucket to store everybody's
messages, will be very hard to query... instead, if you atomize it like
"Messages_Rohman", "Messages_OtherUser", etc... you will have less data
on each bucket and queries could be faster and the MapReduce could be an
Option for OnDemand data.

Rohman 

On Tue, 26 Jul 2011 20:12:08 -0700,
Kev Burns wrote: 

> sorry i meant to post this to the list
> 
> - Kev
>
c: +001 (650) 521-7791
> 
> On Tue, Jul 26, 2011 at 8:11 PM, Kev Burns
wrote:
> 
>> Francisco - How's performance on riak-ql?
>> 
>> The
problem I see with riak-ql and Antonio's thing is that they're
invariably going to be slow.
>> Javascript Map/Reduce over an entire
bucket is just not suitable for inline requests.
>> 
>> Take PodCrazy

>> http://podcrazy.net/ [28]
>> 
>> It's backed entirely by RiakSearch
and memcached.
>> That episode listing on the homepage is a map/reduce
that calculates popularity based on votes over time.
>> But right now
this simple javascript map/reduce over less than a thousand items takes
about 2 seconds to run.
>> It totally makes sense as a map/reduce
because it's calculating popularity based on several decaying
attributes.
>> But it has to happen in a background process.
>> 
>> The
site is powered by this port of Ripple to PHP
>>
http://ripple-php.hackyhack.net/test/?test=document [29]
>> 
>> Right
now ripple-php has remained pretty basic and for good reason.
>> I've
put off creating something more full-featured until secondary indexes
makes it into master.
>> The shape of the native secondary index
mechanism will heavily influence the design of any Riak ODM.
>> 
>>
Lastly, in my experience Riak Search is not very memory efficient as a
non-fulltext index mechanism.
>> Also sort of useless without support
for ranges on anything other than keys.
>> You wind up selecting limit
0,999 and doing the slice yourself.
>> 
>> I suppose I see the value of
a tool like riak-ql for reporting.
>> And I imagine this sort of tool
will continue to be useful to add the sort of features that Secondary
Indexes will not support.
>> And I also imagine a tool like this would
be faster if implemented in Erlang. 
>> 
>> - Kev
>> c: +001 (650)
521-7791
>> 
>> On Tue, Jul 26, 2011 at 7:35 PM, Antonio Rohman
Fernandez wrote:
>> 
>>> thanks for porting it to Google Docs, even
seems the text got a little compressed in there, too cluttered.
>>> hope
it can help somebody. 
>>> 
>>> Rohman 
>>> 
>>> On Tue, 26 Jul 2011
19:21:36 -0700, Kev Burns wrote: 
>>> 
>>>> Here's a virus-free version
of Antonio's slide deck (Google Docs)
>>>>
https://docs.google.com/present/view?id=dhpxng6q_51gdj6r9wn [21]
>>>>

>>>> - Kev
>>>> c: +001 (650) 521-7791
>>>> 
>>>> On Tue, Jul 26, 2011
at 6:23 PM, Antonio Rohman Fernandez wrote:
>>>> 
>>>>> for PHP you can
take a look at this slides i made, is about "phpCloud Framework" a new
PHP5 MVC framework i'm building with Riak integration in place : ) is
based on CakePHP that borrows heavily on Ruby on Rails.
>>>>> You can
download the slides on this address ( seems the file is too big for the
distribution list as my last mail couldn't be sent ):
>>>>> 
>>>>>
http://mahalostudio.com/Riak_phpCloud.pptx [13]
>>>>> 
>>>>> Rohman

>>>>> 
>>>>> -- 
>>>>> 
>>>>> [14]
>>>>> 
>>>>> ANTONIO ROHMAN
FERNANDEZ
>>>>> CEO, Founder & Lead Engineer
>>>>>
rohman at mahalostudio.com [15] 
>>>>> PROJECTS
>>>>> MaruBatsu.es
[16]
>>>>> PupCloud.com [17]
>>>>> Wedding Album [18] 
>>>>> 
>>>>> On
Tue, 26 Jul 2011 20:00:27 -0400, Jonathan Langevin wrote: 
>>>>> 
>>>>>>
Looks interesting, but doesn't appear very intuitive (at least, to a PHP
dev) 
>>>>>> 
>>>>>> [9]
>>>>>> 
>>>>>> _JONATHAN LANGEVIN_
>>>>>>
Systems Administrator 
>>>>>> 
>>>>>> LOOM INC.
>>>>>> Wilmington, NC:
(910) 241-0433 - jlangevin at loomlearning.com [10] - www.loomlearning.com
[11] - Skype: intel352 
>>>>>> 
>>>>>> On Mon, Jul 25, 2011 at 9:40 AM,
francisco treacy wrote: 
>>>>>> 
>>>>>>> It's awesome for ad-hoc
querying, at least. An example can better explain.
>>>>>>> 
>>>>>>>
Consider this:
>>>>>>> 
>>>>>>> db.add('users').map('query', '.address
.street where .weight:expr(x !.expired').run() 
>>>>>>> 
>>>>>>> as
opposed to:
>>>>>>> 
>>>>>>> db.add('users').map(function(v) {
>>>>>>> v
= Riak.mapValuesJson(v)[0];
>>>>>>> var result = [];
>>>>>>> if
((v.weight < 180 || v.exempt) && v.acl && v.acl.state === '1101'
>>>>>>>
&& !v.expired) {
>>>>>>> if (v.address) {
>>>>>>>
result.push(v.address.street);
>>>>>>> }
>>>>>>> }
>>>>>>> return
result;
>>>>>>> }).run()
>>>>>>> 
>>>>>>> riak-ql is basically adding
some query sugar (where, &&) on top of
>>>>>>> JSONSelect... which you
can try it out here:
>>>>>>> http://jsonselect.org/#tryit [1]
>>>>>>>

>>>>>>> 2011/7/25 Mark Phillips :
>>>>>>> 
>>>>>>>> Hey
Francisco,
>>>>>>> >
>>>>>>> > I for one would be interested in learning
some more specifics on how
>>>>>>> > you're using it. I suspect others
might be, too...
>>>>>>> >
>>>>>>> > Mark
>>>>>>> >
>>>>>>> > On Sat,
Jul 23, 2011 at 4:40 PM, francisco treacy
>>>>>>> > wrote:
>>>>>>> >>
Hey all,
>>>>>>> >>
>>>>>>> >> Just wondering... is anyone using, or
have tried out riak-ql?
>>>>>>> >> https://github.com/frank06/riak-ql
[4]
>>>>>>> >>
>>>>>>> >> Not because I developed it -- but I'm
regularly making use of it and I
>>>>>>> >> think it kicks ass.
Especially in the repl in combo with riak-js.
>>>>>>> >>
>>>>>>> >> What
do you guys think?
>>>>>>> >>
>>>>>>> >> Francisco
>>>>>>> >>
>>>>>>> >>
ps: really curious/excited about the upcoming Secondary Indices
functionality
>>>>>>> >>
>>>>>>> >>
_______________________________________________
>>>>>>> >> riak-users
mailing list
>>>>>>> >> riak-users at lists.basho.com [5]
>>>>>>> >>
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
[6]
>>>>>>> >>
>>>>>>> >
>>>>>>> 
>>>>>>>
_______________________________________________
>>>>>>> riak-users
mailing list
>>>>>>> riak-users at lists.basho.com [7]
>>>>>>>
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
[8]
>>>>> 
>>>>> _______________________________________________
>>>>>
riak-users mailing list
>>>>> riak-users at lists.basho.com [19]
>>>>>
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
[20]
>>> 
>>> -- 
>>> 
>>> [23]
>>> 
>>> ANTONIO ROHMAN FERNANDEZ
>>>
CEO, Founder & Lead Engineer
>>> rohman at mahalostudio.com [24] 
>>>
PROJECTS
>>> MaruBatsu.es [25]
>>> PupCloud.com [26]
>>> Wedding Album
[27]

-- 

 		 [32]

 ANTONIO ROHMAN FERNANDEZ
CEO, Founder & Lead
Engineer
rohman at mahalostudio.com [33] 		 
 PROJECTS
MaruBatsu.es
[34]
PupCloud.com [35]
Wedding Album [36] 

   

Links:
------
[1]
http://jsonselect.org/#tryit
[2] mailto:mark at basho.com
[3]
mailto:francisco.treacy at gmail.com
[4]
https://github.com/frank06/riak-ql
[5]
mailto:riak-users at lists.basho.com
[6]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
[7]
mailto:riak-users at lists.basho.com
[8]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
[9]
http://www.loomlearning.com/
[10] mailto:jlangevin at loomlearning.com
[11]
http://www.loomlearning.com/
[12] mailto:francisco.treacy at gmail.com
[13]
http://mahalostudio.com/Riak_phpCloud.pptx
[14]
http://mahalostudio.com/
[15] mailto:rohman at mahalostudio.com
[16]
http://marubatsu.es/
[17] http://pupcloud.com/
[18]
http://wedding.mahalostudio.com/
[19]
mailto:riak-users at lists.basho.com
[20]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
[21]
https://docs.google.com/present/view?id=dhpxng6q_51gdj6r9wn
[22]
mailto:rohman at mahalostudio.com
[23] http://mahalostudio.com
[24]
mailto:rohman at mahalostudio.com
[25] http://marubatsu.es
[26]
http://pupcloud.com
[27] http://wedding.mahalostudio.com
[28]
http://podcrazy.net/
[29]
http://ripple-php.hackyhack.net/test/?test=document
[30]
mailto:rohman at mahalostudio.com
[31] mailto:kevburnsjr at gmail.com
[32]
http://mahalostudio.com
[33] mailto:rohman at mahalostudio.com
[34]
http://marubatsu.es
[35] http://pupcloud.com
[36]
http://wedding.mahalostudio.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110727/6a4dfe94/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: blocked.gif
Type: image/gif
Size: 118 bytes
Desc: not available
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110727/6a4dfe94/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: blocked.gif
Type: image/gif
Size: 118 bytes
Desc: not available
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110727/6a4dfe94/attachment-0001.gif>


More information about the riak-users mailing list