Map/Red Function Cache Corruption?

Mathias Meyer mathias at basho.com
Tue Jul 5 03:05:02 EDT 2011


Eric,

are you by any chance still running Riak 0.14.1? There was a bug showing the same symptoms you're describing, which was fixed in the recent 0.14.2 release.

Mathias Meyer
Developer Advocate, Basho Technologies


On Samstag, 2. Juli 2011 at 14:40, Eric Stevens wrote:

> I've been struggling with this for a few days now. I have a pair of related Map/Reduce queries. Executed individually they produce the expected results, but when executed in parallel (simultaneous XHR calls from the browser), their results tend to get mixed up with each other; I see values from both mixed together in a single result. 
> 
> Once that corruption happens, even the calls executed individually can end up with cross-pollinated results. It seems to be related to the caching of the anonymous javascript functions used, as adding a space to the function declaration clears the issue up. Taking that space back out causes the problem again. 
> 
> I can confirm that the corruption can become persistent by using a line such as (PHP client):
> ejsLog('/tmp/mapred/reduce_" . join('.', explode('\\', get_class($this))) . ".log', JSON.stringify(values));
> 
> After the corruption happens, if I clear the log path, execute just the one map/red operation, I will see log entries appear for both classes (the class name being encoded as part of the function declaration). The problem seems to happen for both map and reduce operations, though I have not specifically noticed cross-contamination between map and reduce functions, just map-for-map and reduce-for-reduce. 
> 
> Adding this as the first line of the function declaration seems to solve the problem, but this is obviously something intended to defeat caching, so I'm sure there's a performance cost to this (note, just get_class($this) is not sufficient, cross-contamination can still happen, each call needs to be unique call-to-call, thus the mt_rand()). 
> "//".get_class($this).' '.mt_rand()."
> 
> 
> I noticed Francisco Treacy had this same problem back in September: http://riak-users.197444.n3.nabble.com/reduce-headaches-td1524860.html but I didn't see any resolution there. Is there a potential longer term cost to my cachebusting, such as memory exhaustion (i.e. is there any garbage collection on the function cache)? 
> 
> Any help would be sincerely appreciated! 
> 
> -Eric 
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com (mailto:riak-users at lists.basho.com)
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com






More information about the riak-users mailing list