Split index with Riak 2.0 git (jan 15th) on a single dev node cluster

Eric Redmond eredmond at basho.com
Thu Jan 23 18:07:20 EST 2014


Rob,

The one second wait is because yokozuna is the glue code (putting it very, very simply) between a Riak cluster and distributed Solr instances. When you write an object to Riak, yokozuna asynchronously fires off an update to the Solr service. Solr is, by default, configured to soft commit writes every 1 second, and hard commit every 30 seconds[1]. These values are set in the Solr configuration, based on suggested best practiced. You can configure solr.xml to reduce that delay, but the payoff isn't generally worth the overhead.

The benefit of an asynchronous index update is that you get the predictable performance of Riak, with an eventually consistent searchable index. The alternative is to lock a Riak put/delete while Solr updates, which means you'll wait up to a second for confirmation for a simple write.  We may eventually add such back pressure, but it's barely on our radar, since at the moment, atomic object/index updating is not of practical concern to most Riak users.  If you have a specific requirement for such back pressure, please let us know and we may have some ideas for you.

Another benefit to separating the write from indexing is that it allows us to leverage an index-specific AAE service. This means that if your dataset and index ever get out of sync, or your index corrupted, the index AAE can repair the discrepancy. If you've ever run Solr or Elastic Search against another datastore, you'll be familiar with having to repair indexes on occasion (whether by a server going down, other system errors, bit rot, etc). With Riak Search (yokozuna) this is handled automatically for you.

Hope that helps,
Eric

http://wiki.apache.org/solr/NearRealtimeSearch


On Jan 23, 2014, at 5:41 PM, Rob Speer <rob at luminoso.com> wrote:

> I'm still interested in the question as it applies to Yokozuna. Stable full-text search in Riak will be very important to my company, so I'd want to know what the equivalent behaviors are for Yokozuna queries. Do you have to wait an unspecified amount of time between writing a document and querying it via Yokozuna? Is there a way to know when it is ready?
> 
> 
> On Thu, Jan 23, 2014 at 10:53 AM, Luke Bakken <lbakken at basho.com> wrote:
> Hi Rob,
> 
> I believe Ryan meant to wait a second to do a Yokozuna search, not a
> general Riak K/V operation.
> 
> There is more information about "read your own writes" here:
> http://basho.com/tag/configurable-behaviors/
> --
> Luke Bakken
> CSE
> lbakken at basho.com
> 
> 
> On Wed, Jan 22, 2014 at 11:36 AM, Rob Speer <rob at luminoso.com> wrote:
> >> 5. Did you wait at least 1 second before running the queries?
> >
> > I'm not the original poster but I'm now wondering what this question means.
> > Under what circumstances do you have to wait 1 second before query results
> > are available?
> >
> > We want to always be able to run tests on our database rapidly, which
> > includes loading data and then immediately querying to make sure the correct
> > data is there. I know we've had some tests where we were not able to read
> > our own writes, but I thought those were fixed by making sure we used vector
> > clocks correctly. Is there a situation where you have to wait for an
> > unspecified amount of time before you can read your writes?
> >
> > _______________________________________________
> > riak-users mailing list
> > riak-users at lists.basho.com
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >
> 
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20140123/419391bb/attachment.html>


More information about the riak-users mailing list