Escape characters in MapReduce?

Kevin Burton rkevinburton at charter.net
Sun Feb 10 20:18:04 EST 2013


This is just like the canonical ‘word count’ MapReduce’ I just want to count the number of departments in the bucket. The ‘!=’ is to skip the values of Department that are the empty string.

 

From: Alexander Sicular [mailto:siculars at gmail.com] 
Sent: Sunday, February 10, 2013 7:14 PM
To: Kevin Burton
Cc: <riak-users at lists.basho.com>
Subject: Re: Escape characters in MapReduce?

 

What is p.Department not supposed to equal?





@siculars

http://siculars.posterous.com

 

Sent from my iRotaryPhone


On Feb 10, 2013, at 18:36, "Kevin Burton" <rkevinburton at charter.net> wrote:

I have a query that is made from a CorrugatedIron client that looks like:

 

                IRiakClient riakClient = cluster.CreateClient();

                var query = new RiakMapReduceQuery()

                    .Inputs(productBucketName)

                    .MapJs(m => m.Source(@"function(v,d,a) {" +

                        "var p = JSON.parse(v.values[0].data);" + 

                        "var r = [];" +

                        "if(p.Department != '') {" +

                        "var o = {};" +

                        "o[p.Department] = 1;" +

                        "r.push(o);" +

                        "}" +

                        "return r;" +

                        "}"))

                    .ReduceJs(m => m.Source(@"function(v,d,a) {" +

                        "var r = {};" +

                        "for(var i in v) {" +

                        "  for(var w in v[i]) {" +

                        "    if(w in r) r[w] += v[i][w];" +

                        "    else r[w] = v[i][w];" +

                        "  }" +

                        "}" +

                        "return [r];" +

                        "}")

                        .Keep(true));

 

This query returns the error:

 

Query failed with Riak returned an error. Code '0'. Message: {"phase":"listkeys","error":"{badmatch,{'EXIT',timeout}}","input":"{cover,[],<<\"buyseasons-products\">>}","type":"error","stack":"[{riak_core_vnode_proxy,call,2,[{file,\"src/riak_core_vnode_proxy.erl\"},{line,52}]},{riak_pipe_vnode,queue_work_send,4,[{file,\"src/riak_pipe_vnode.erl\"},{line,331}]},{riak_pipe_vnode,queue_work_erracc,6,[{file,\"src/riak_pipe_vnode.erl\"},{line,279}]},{riak_kv_pipe_listkeys,keysend,4,[{file,\"src/riak_kv_pipe_listkeys.erl\"},{line,112}]},{riak_kv_pipe_listkeys,keysend_loop,3,[{file,\"src/riak_kv_pipe_listkeys.erl\"},{line,89}]},{riak_kv_pipe_listkeys,process,3,[{file,\"src/riak_kv_pipe_listkeys.erl\"},...]},...]"} - CommunicationError

 

I can read the error and it is a “badmatch” but the rest of the details I don’t know how to use. The purpose of these query is to count all of the different Departments in the database. I was wondering if this requires that the Department name be escaped somehow. One of the department names is ‘Accessories & Makeup’,  ‘Decorations & Props’,  or ‘Hats, Wigs & Masks’. Does the ‘&’ need to be escaped when putting it into the database (this string is a property that is part of the object that is JSON serialized)? Anything else? If it needs to be escaped and I am using this value as a secondary index should I set the index before or after the escape is done? Could this be the source of a “badmatch”?

 

_______________________________________________
riak-users mailing list
riak-users at lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130210/b38d6746/attachment.html>


More information about the riak-users mailing list