0.14 and search schema

Gary William Flake gary at flake.org
Mon Feb 21 01:56:19 EST 2011

I've been pulling my hair out over new behaviors with search-schema,
and I am wondering if I've been simply doing it wrong.  Here's the
issue: search-schema seem extremely brittle compared to the rest of
the system in that one misstep requires that you effectively blow away
your whole DB and try again.  Some specific examples:

1. Defaults don't seem to mean anything now in that if you add a new
field to what you are storing, then riak will throw up on insert
because the field isn't in the schema.

2. But if you add the new field to the schema, then it breaks all of
your old records (e.g., they can't be deleted until you change the
schema back or turn off hooks).

3. The semantics are non-intuitive (at least to me).  I had thought
that {skip, true} would mean "this is data that I want but don't index
over it".  Instead, it appears to mean "riak will not index it, and if
you do a search, we will not provide this field value in the results
even though it exists".

4. To make it even more challenging, on ubuntu "search-cmd set-schema
..." ignores the last argument.  It seems to expect the schema to
already exist in /var/lib/riak/scripts and I found that if I
forcefully move and chmod the schema, then I can get it to stick.

5. But, BTW, /var/lib/riak doesn't exist as part of 0.14's deb
package, so you better know to add it yourself.

Anyhow, am I doing something wrong?  Or is how it is supposed to be?
Does anyone have any tricks for working around this brittleness?

-- GWF

PS - here's a trick that may help you.  Define a field called "future"
and make set up the standard analyzer in the search schema.  Then,
when you are tempted to add a new field, don't.  Instead, force your
new "field:val" bits into future instead.  This way, you can get away
with not having to change the schema nor do you need to add a new
field.  I am hoping that this trick will help me keep a riak instance
stable long enough to work through migration issues for when I do need
to change the schema.

More information about the riak-users mailing list