annoying javascript problem in Reduce phases

Alexander Sicular siculars at gmail.com
Thu Mar 10 09:03:55 EST 2011


Echo what Sean said. Also I have a post on my blog about pagination  
and riak. As Sean said, it is basically a reduce(sort) , reduce(slice)  
to get it working.

-Alexander.

@siculars on twitter
http://siculars.posterous.com

Sent from my iPhone

On Mar 10, 2011, at 8:26, Antonio Rohman Fernandez <rohman at mahalostudio.com 
 > wrote:

> Dear Sean,
> thank you for your quick reply.
>
> "Your reduce function can (almost) never assume that it is operating  
> on the entire result set." -> wow! that totally kills my  
> implementation... i wanted to use the Reduce phase to cut the array  
> as a Limit + Pagination ( like LIMIT 10 PAGE 2 in SQL )... but if i  
> don't have the full result set is impossible and useless. I will  
> have to get all the rows and then process it outside Riak for that.
>
> "you must structure your reduce function such that it can receive as  
> inputs anything that it outputs" -> what do you mean with that? can  
> i merge Reduce phases? any example?
>
> i'm building a php MVC framework in top of Riak, and i would like to  
> clone from my SQL implementation as much functionality as i can, so  
> people can easily use Riak as if they were using MySQL/PostgreSQL/ 
> etc without noticing.
> thanks.
>
> Rohman
>
>
> 	 	Antonio Rohman Fernandez
> CEO, Founder & Lead Engineer
> rohman at mahalostudio.com
>  	Projects
> MaruBatsu.es
> PupCloud.com
> Wedding Album
>
>
> On Mar 10, 2011, at 8:50 PM, Sean Cribbs wrote:
>
>> Antonio,
>>
>> Sorry it's a little buried in the documentation, but be aware that  
>> Reduce phases *may* be called more than once (CouchDB calls this  
>> rereduce).  What you experience in the last example is a  
>> demonstration of that: you must structure your reduce function such  
>> that it can receive as inputs anything that it outputs.  The array/ 
>> list that you return as the output will be concatenated with more  
>> values on the subsequent applications of the function.  Your reduce  
>> function can (almost) never assume that it is operating on the  
>> entire result set.
>>
>> Sean Cribbs <sean at basho.com>
>> Developer Advocate
>> Basho Technologies, Inc.
>> http://basho.com/
>>
>> On Mar 10, 2011, at 5:32 AM, Antonio Rohman Fernandez wrote:
>>
>>> Dear All,
>>> I'm having a pretty annoying Javascript problem and would like to  
>>> know if is a bug (most likely) or if somebody has a solution.
>>>
>>> This is the sample data i'm playing with:
>>> bucket -> 'testbucket'
>>> key00001:{"name":"antonio","dob":1981,"country":"Spain"}
>>> key00002:{"name":"rohman","dob":1982,"country":"Taiwan"}
>>> key00003:{"name":"fernandez","dob":1983,"country":"US"}
>>> key00004:{"name":"lee","dob":1984,"country":"Japan"}
>>> key00005:{"name":"bruce","dob":1985,"country":"China"}
>>>
>>> MAP ONLY: {"inputs":"testbucket","query":[{"map": 
>>> {"language":"javascript","source":"function(v,k,a) { return  
>>> [v.values[0].data]; }"}}]}
>>>
>>> The Mapping phase ( nothing special, just gives back the values of  
>>> all keys in the bucket ) gives the correct data, an array of all  
>>> my key's values:
>>> ["{\"name\":\"rohman\",\"dob\":1982,\"country\":\"Taiwan\"}",
>>> "{\"name\":\"bruce\",\"dob\":1985,\"country\":\"China\"}",
>>> "{\"name\":\"fernandez\",\"dob\":1983,\"country\":\"US\"}",
>>> "{\"name\":\"antonio\",\"dob\":1981,\"country\":\"Spain\"}",
>>> "{\"name\":\"lee\",\"dob\":1984,\"country\":\"Japan\"}"]
>>>
>>> REDUCE (try1:OK): {"inputs":"testbucket","query":[{"map": 
>>> {"language":"javascript","source":"function(v,k,a) { return  
>>> [v.values[0].data]; }"}},{"reduce":  
>>> {"language":"javascript","source":"function(v,a) { return v; }"}}]}
>>>
>>> This kinda useless Reduce phase returns the data as it is, even  
>>> the order is reversed... seems ok:
>>> ["{\"name\":\"lee\",\"dob\":1984,\"country\":\"Japan\"}",
>>> "{\"name\":\"fernandez\",\"dob\":1983,\"country\":\"US\"}",
>>> "{\"name\":\"rohman\",\"dob\":1982,\"country\":\"Taiwan\"}",
>>> "{\"name\":\"antonio\",\"dob\":1981,\"country\":\"Spain\"}",
>>> "{\"name\":\"bruce\",\"dob\":1985,\"country\":\"China\"}"]
>>>
>>> REDUCE (try2:OK): {"inputs":"testbucket","query":[{"map": 
>>> {"language":"javascript","source":"function(v,k,a) { return  
>>> [v.values[0].data]; }"}},{"reduce":  
>>> {"language":"javascript","source":"function(v,a) { return [v 
>>> [1]]; }"}}]}
>>>
>>> This Reduce phase just return the 2nd item array[1] of the  
>>> result... seems ok too!:
>>> ["{\"name\":\"fernandez\",\"dob\":1983,\"country\":\"US\"}"]
>>>
>>> REDUCE (try3:!!!!!): {"inputs":"testbucket","query":[{"map": 
>>> {"language":"javascript","source":"function(v,k,a) { return  
>>> [v.values[0].data]; }"}},{"reduce":  
>>> {"language":"javascript","source":"function(v,a) { return [v 
>>> [3]]; }"}}]}
>>>
>>> This Reduce phase SHOULD return the 4th item of the array... but  
>>> returns NULL!!!!:
>>> [null] <-- WTF... works for [v[0]], [v[1]], [v[2]] but fails from  
>>> [v[3]]...
>>>
>>> REDUCE (try4:!!!!!): {"inputs":"testbucket","query":[{"map": 
>>> {"language":"javascript","source":"function(v,k,a) { return  
>>> [v.values[0].data]; }"}},{"reduce":  
>>> {"language":"javascript","source":"function(v,a) { return  
>>> [v]; }"}}]}
>>>
>>> This Reduce phase returns the data inside an array... but with a  
>>> wrong patter [[1,1,1,[1,1]]]!!!:
>>> [
>>> ["{\"name\":\"lee\",\"dob\":1984,\"country\":\"Japan\"}",
>>> "{\"name\":\"fernandez\",\"dob\":1983,\"country\":\"US\"}",
>>> "{\"name\":\"rohman\",\"dob\":1982,\"country\":\"Taiwan\"}",
>>> ["{\"name\":\"antonio\",\"dob\":1981,\"country\":\"Spain\"}",   
>>> <---- See the weird extra array?
>>> "{\"name\":\"bitch\",\"dob\":1985,\"country\":\"China\"}"]
>>> ]
>>> ]
>>>
>>> that is transformed to this:
>>>
>>> Array (
>>>   [0] => {"name":"lee","dob":1984,"country":"Japan"}
>>>   [1] => {"name":"fernandez","dob":1983,"country":"US"}
>>>   [2] => {"name":"rohman","dob":1982,"country":"Taiwan"}
>>>   [3] => Array (
>>>     [0] => {"name":"antonio","dob":1981,"country":"Spain"}
>>>     [1] => {"name":"bruce","dob":1985,"country":"China"}
>>>   )
>>> )
>>>
>>> I don't know what is going on with the javascript Reduce phase...  
>>> but is killing me...
>>> return v --> OK
>>> return [v[0]] --> OK
>>> return [v[1]] --> OK
>>> return [v[2]] --> OK
>>> return [v[3]] --> ERROR ( and so on )
>>>
>>> any ideas? thanks
>>> Rohman
>>>
>>>
>>> 	
>>> Antonio Rohman Fernandez
>>> CEO, Founder & Lead Engineer
>>> rohman at mahalostudio.com	 	Projects
>>> MaruBatsu.es
>>> PupCloud.com
>>> Wedding Album
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> riak-users at lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110310/d9c1a940/attachment.html>


More information about the riak-users mailing list