Riak Map / Reduce / Map capacity limit reached at 20000 keys

claudef at br.ibm.com claudef at br.ibm.com
Mon May 21 13:02:56 EDT 2012


Dear colleagues, 

 During my pilot testing I've got an error message during a text pattern 
search operation coded as a Riak Client "map" logic. This error appeared 
at a volume of approximately 20.000 keys, each with an 1 KB free text data 
block associated. The Map function was coded in JavaScript. Here the 
source: 

query.map("""function(v) { var str = '"""+search_string+"""'; var data = 
JSON.parse(v.values[0].data); if (v.values[0].data.toString().search(str) 
!= -1) { return [[v.key, data]]; } return []; }""")
query.reduce("function(v) { var counteri = 0;  for(var i in v) {counteri 
+= 1;} return [['counter_found', counteri]]; } ")

The error shown is the following : [preflist_exhausted]

zbra:/opt/python/myprojects/riak # python pilot3.py
starts at : 2012-05-21 07:35:49.070470
How many keys in the bucket: 19760
Riak client exception: Error running MapReduce operation. Status: 500 : 
{"phase":0,"error":"[preflist_exhausted]","input":"{ok,{r_object,<<\"ramtest\">>,<<\"XDHt25887344\">>,[{r_content,{dict,6,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[[<<\"Links\">>]],[],[],[],[],[],[],[],[[<<\"content-type\">>,97,112,112,108,105,99,97,116,105,111,110,47,106,115,111,110],[<<\"X-Riak-VTag\">>,53,86,72,48,120,77,53,99,114,86,111,77,77,54,109,114,79,70,54,52,49,109]],[[<<\"index\">>]],[],[[<<\"X-Riak-Last-Modified\">>|{1337,595548,374416}]],[],[[<<\"X-Riak-Meta\">>]]}}},<<\"{\"is_valid\": 
true, \"cEDO\": 
\"GQZumfdWBCfX...\">>}],...},...}","type":"forward_preflist","stack":"[]"}

Any suggestions how to improve the pattern scan capability in the in the 
RiakClient  map function, or how enhance the JavaScript code for better 
capacity? 
Are there any servers setup tuning options to adjust? 

Thanks in advance for your support. 

Regards, 
Claude

Claude Falbriard 
Certified IT Specialist L2 - Middleware
AMS Hortolândia / SP - Brazil
phone:    +55 19 9837 0789
cell:         +55 13 8117 3316
e-mail:    claudef at br.ibm.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20120521/81b8ff9c/attachment.html>


More information about the riak-users mailing list