Inconsistent map/reduce results

Keith Dreibelbis kdreibel at gmail.com
Thu Mar 10 17:16:37 EST 2011


Hi riak-users,

I'm trying to do a map/reduce query from java on a 0.13 server, and get
inconsistent results.  What I'm doing should be pretty simple.  I'm hoping
someone will notice an obvious error in here, or have some insight:

This is an automated test.  I'm doing a simple query where I'm trying to get
the keys for records with a certain field value.  In SQL it would look like
"SELECT id FROM table WHERE age = '32'".  In java I'm invoking it like this:

  MapReduceResponse r = riak.mapReduceOverBucket(getBucket())
        .map(JavascriptFunction.anon(func), true)
             .submit();

where riak is a RiakClient, getBucket() returns the name of the bucket, and
func is a string that looks like:

function(value, keyData, arg) {
       var data = Riak.mapValuesJson(value)[0];
       if(data.age == "32")
         return [value.key];
      else
         return [];
   }

No reduce phase.  All entries in the example bucket are json and have an age
field.  This initially works correctly, it gets back the matching records as
expected.  It also works in curl.  It's an automated test, so each time I
run this, it is using a different bucket.  After about a dozen queries, this
starts to fail.  It returns an empty result, when it should have found
records.  It fails in curl at the same time.

I initially suspected this might have something to do with doing map reduce
too soon after writing, and the write not being available on all nodes.
 However, I changed the bucket schema entries for w,r,rw,dw from "quorum" to
"all", and this still happens (is there another bucket setting I missed?).
In addition, I only have 3 nodes (I'm using the dev123 example), and am
running curl long enough afterwards.

Here's the strange part that makes me suspicious.  If I make insignificant
changes to the query, for example change the double quotes to single quotes,
add whitespace or extra parentheses, etc, then it suddenly works again.  It
will work on an existing bucket, and on subsequent tests, but again only
about a dozen times before it starts failing again. Same behavior in curl.
 This makes me suspect that the server is doing some incorrect caching
around this js function, based on the function string.

Any explanation about what's going on?


Keith
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110310/04e23714/attachment.html>


More information about the riak-users mailing list