Riak Search and Yokozuna Backup Strategy

Elias Levy fearsome.lucidity at gmail.com
Mon Jan 27 16:02:49 EST 2014


On Mon, Jan 27, 2014 at 12:29 PM, Ryan Zezeski <rzezeski at basho.com> wrote:

>  Any comments on the backup strategy for Yokozuna?  Will it make use of
>> Solr's Replication Handler, or something more lower level?  Will the node
>> need to be offline to backup it up?
>>
>
> There is no use of any Solr replication code--at all.
>

I knew that.   I mention the Solr Replication Handler because its one way
to make a backup<https://cwiki.apache.org/confluence/display/solr/Backing+Up>of
Solr, even when there are no replicas of the index.  It can be used to
have Solr create a backup by creating hard links to the index files.

Another issue is that various parts of Riak that write to disk may not be
> snapshot safe. It's already been discussed how leveldb isn't. I'm willing
> to bet Lucene isn't either. Any case where a logical operation requires
> multiple filesystem writes you have to worry about the snapshot occurring
> in the middle of the logical operation. I have no idea how Lucene would
> deal with snapshots that occur at the wrong time. I'm unsure how good it is
> at detecting, and more importantly, recovering from corruption.
>

It appears that the recommended way to make a Lucene backup is to make use
of SnapshotDeletionPolicy<http://lucene.apache.org/core/4_4_0/core/org/apache/lucene/index/SnapshotDeletionPolicy.html>to
take a snapshot, backup all the files the snapshot references, and
then
to release the snapshot.

So it would appear to do it properly, we'd need some support from Yokozuna
to take the snapshot, return a list of files to backup or back them up
itself (hard links?), and then to allow an application to signal it to
release the snapshot or release it itself if its doing the backup.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20140127/1bd4bf55/attachment.html>


More information about the riak-users mailing list