Map/Red Function Cache Corruption?
mightye at gmail.com
Wed Jul 6 21:25:13 EDT 2011
Yep, our sysadmin admitted he's behind on upgrades, we'll get it upgraded
and see if that solves the problem. Thanks Mathias!
On Tue, Jul 5, 2011 at 3:05 AM, Mathias Meyer <mathias at basho.com> wrote:
> are you by any chance still running Riak 0.14.1? There was a bug showing
> the same symptoms you're describing, which was fixed in the recent 0.14.2
> Mathias Meyer
> Developer Advocate, Basho Technologies
> On Samstag, 2. Juli 2011 at 14:40, Eric Stevens wrote:
> > I've been struggling with this for a few days now. I have a pair of
> related Map/Reduce queries. Executed individually they produce the expected
> results, but when executed in parallel (simultaneous XHR calls from the
> browser), their results tend to get mixed up with each other; I see values
> from both mixed together in a single result.
> > Once that corruption happens, even the calls executed individually can
> end up with cross-pollinated results. It seems to be related to the caching
> function declaration clears the issue up. Taking that space back out causes
> the problem again.
> > I can confirm that the corruption can become persistent by using a line
> such as (PHP client):
> > ejsLog('/tmp/mapred/reduce_" . join('.', explode('\\', get_class($this)))
> . ".log', JSON.stringify(values));
> > After the corruption happens, if I clear the log path, execute just the
> one map/red operation, I will see log entries appear for both classes (the
> class name being encoded as part of the function declaration). The problem
> seems to happen for both map and reduce operations, though I have not
> specifically noticed cross-contamination between map and reduce functions,
> just map-for-map and reduce-for-reduce.
> > Adding this as the first line of the function declaration seems to solve
> the problem, but this is obviously something intended to defeat caching, so
> I'm sure there's a performance cost to this (note, just get_class($this) is
> not sufficient, cross-contamination can still happen, each call needs to be
> unique call-to-call, thus the mt_rand()).
> > "//".get_class($this).' '.mt_rand()."
> > I noticed Francisco Treacy had this same problem back in September:
> http://riak-users.197444.n3.nabble.com/reduce-headaches-td1524860.html but
> I didn't see any resolution there. Is there a potential longer term cost to
> my cachebusting, such as memory exhaustion (i.e. is there any garbage
> collection on the function cache)?
> > Any help would be sincerely appreciated!
> > -Eric
> > _______________________________________________
> > riak-users mailing list
> > riak-users at lists.basho.com (mailto:riak-users at lists.basho.com)
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the riak-users