annoying javascript problem in Reduce phases

Antonio Rohman Fernandez rohman at mahalostudio.com
Thu Mar 10 08:26:05 EST 2011


Dear Sean,
thank you for your quick reply.

"Your reduce function can (almost) never assume that it is operating on the entire result set." -> wow! that totally kills my implementation... i wanted to use the Reduce phase to cut the array as a Limit + Pagination ( like LIMIT 10 PAGE 2 in SQL )... but if i don't have the full result set is impossible and useless. I will have to get all the rows and then process it outside Riak for that.

"you must structure your reduce function such that it can receive as inputs anything that it outputs" -> what do you mean with that? can i merge Reduce phases? any example?

i'm building a php MVC framework in top of Riak, and i would like to clone from my SQL implementation as much functionality as i can, so people can easily use Riak as if they were using MySQL/PostgreSQL/etc without noticing.
thanks.

Rohman


	 	Antonio Rohman Fernandez
CEO, Founder & Lead Engineer
rohman at mahalostudio.com
 	Projects
MaruBatsu.es
PupCloud.com
Wedding Album
 

On Mar 10, 2011, at 8:50 PM, Sean Cribbs wrote:

> Antonio,
> 
> Sorry it's a little buried in the documentation, but be aware that Reduce phases *may* be called more than once (CouchDB calls this rereduce).  What you experience in the last example is a demonstration of that: you must structure your reduce function such that it can receive as inputs anything that it outputs.  The array/list that you return as the output will be concatenated with more values on the subsequent applications of the function.  Your reduce function can (almost) never assume that it is operating on the entire result set.
> 
> Sean Cribbs <sean at basho.com>
> Developer Advocate
> Basho Technologies, Inc.
> http://basho.com/
> 
> On Mar 10, 2011, at 5:32 AM, Antonio Rohman Fernandez wrote:
> 
>> Dear All,
>> I'm having a pretty annoying Javascript problem and would like to know if is a bug (most likely) or if somebody has a solution.
>> 
>> This is the sample data i'm playing with:
>> bucket -> 'testbucket'
>> key00001:{"name":"antonio","dob":1981,"country":"Spain"}
>> key00002:{"name":"rohman","dob":1982,"country":"Taiwan"}
>> key00003:{"name":"fernandez","dob":1983,"country":"US"}
>> key00004:{"name":"lee","dob":1984,"country":"Japan"}
>> key00005:{"name":"bruce","dob":1985,"country":"China"}
>> 
>> MAP ONLY: {"inputs":"testbucket","query":[{"map":{"language":"javascript","source":"function(v,k,a) { return [v.values[0].data]; }"}}]}
>> 
>> The Mapping phase ( nothing special, just gives back the values of all keys in the bucket ) gives the correct data, an array of all my key's values:
>> ["{\"name\":\"rohman\",\"dob\":1982,\"country\":\"Taiwan\"}",
>> "{\"name\":\"bruce\",\"dob\":1985,\"country\":\"China\"}",
>> "{\"name\":\"fernandez\",\"dob\":1983,\"country\":\"US\"}",
>> "{\"name\":\"antonio\",\"dob\":1981,\"country\":\"Spain\"}",
>> "{\"name\":\"lee\",\"dob\":1984,\"country\":\"Japan\"}"]
>> 
>> REDUCE (try1:OK): {"inputs":"testbucket","query":[{"map":{"language":"javascript","source":"function(v,k,a) { return [v.values[0].data]; }"}},{"reduce": {"language":"javascript","source":"function(v,a) { return v; }"}}]}
>> 
>> This kinda useless Reduce phase returns the data as it is, even the order is reversed... seems ok:
>> ["{\"name\":\"lee\",\"dob\":1984,\"country\":\"Japan\"}",
>> "{\"name\":\"fernandez\",\"dob\":1983,\"country\":\"US\"}",
>> "{\"name\":\"rohman\",\"dob\":1982,\"country\":\"Taiwan\"}",
>> "{\"name\":\"antonio\",\"dob\":1981,\"country\":\"Spain\"}",
>> "{\"name\":\"bruce\",\"dob\":1985,\"country\":\"China\"}"]
>> 
>> REDUCE (try2:OK): {"inputs":"testbucket","query":[{"map":{"language":"javascript","source":"function(v,k,a) { return [v.values[0].data]; }"}},{"reduce": {"language":"javascript","source":"function(v,a) { return [v[1]]; }"}}]}
>> 
>> This Reduce phase just return the 2nd item array[1] of the result... seems ok too!:
>> ["{\"name\":\"fernandez\",\"dob\":1983,\"country\":\"US\"}"]
>> 
>> REDUCE (try3:!!!!!): {"inputs":"testbucket","query":[{"map":{"language":"javascript","source":"function(v,k,a) { return [v.values[0].data]; }"}},{"reduce": {"language":"javascript","source":"function(v,a) { return [v[3]]; }"}}]}
>> 
>> This Reduce phase SHOULD return the 4th item of the array... but returns NULL!!!!:
>> [null] <-- WTF... works for [v[0]], [v[1]], [v[2]] but fails from [v[3]]...
>> 
>> REDUCE (try4:!!!!!): {"inputs":"testbucket","query":[{"map":{"language":"javascript","source":"function(v,k,a) { return [v.values[0].data]; }"}},{"reduce": {"language":"javascript","source":"function(v,a) { return [v]; }"}}]}
>> 
>> This Reduce phase returns the data inside an array... but with a wrong patter [[1,1,1,[1,1]]]!!!:
>> [
>> ["{\"name\":\"lee\",\"dob\":1984,\"country\":\"Japan\"}",
>> "{\"name\":\"fernandez\",\"dob\":1983,\"country\":\"US\"}",
>> "{\"name\":\"rohman\",\"dob\":1982,\"country\":\"Taiwan\"}",
>> ["{\"name\":\"antonio\",\"dob\":1981,\"country\":\"Spain\"}",  <---- See the weird extra array?
>> "{\"name\":\"bitch\",\"dob\":1985,\"country\":\"China\"}"]
>> ]
>> ]
>> 
>> that is transformed to this:
>> 
>> Array (
>>   [0] => {"name":"lee","dob":1984,"country":"Japan"}
>>   [1] => {"name":"fernandez","dob":1983,"country":"US"}
>>   [2] => {"name":"rohman","dob":1982,"country":"Taiwan"}
>>   [3] => Array (
>>     [0] => {"name":"antonio","dob":1981,"country":"Spain"}
>>     [1] => {"name":"bruce","dob":1985,"country":"China"}
>>   )
>> )
>> 
>> I don't know what is going on with the javascript Reduce phase... but is killing me...
>> return v --> OK
>> return [v[0]] --> OK
>> return [v[1]] --> OK
>> return [v[2]] --> OK
>> return [v[3]] --> ERROR ( and so on )
>> 
>> any ideas? thanks
>> Rohman
>> 
>> 
>> 	
>> Antonio Rohman Fernandez
>> CEO, Founder & Lead Engineer
>> rohman at mahalostudio.com	 	Projects
>> MaruBatsu.es
>> PupCloud.com
>> Wedding Album
>> 
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110310/a055a290/attachment.html>


More information about the riak-users mailing list