Riak Search and Yokozuna Backup Strategy

Elias Levy fearsome.lucidity at gmail.com
Mon Jan 27 14:40:34 EST 2014


On Thu, Jan 23, 2014 at 10:25 AM, Joe Caswell <jcaswell at basho.com> wrote:

> Backing up LevelDB data files can be accomplished while the node is
> running if the sst_x directories are backed up in numerical order.  The
> undesirable side effects of that could be duplicated data, inconsistent
> manifest, or incomplete writes, which necessitates running the leveldb
> repair process upon restoration for any vnode backed up while the node was
> running.  Since the data is initially written to the recovery log before
> being appended to level 0, and any compaction operation fully writes the
> data to its new location before removing it from its old location, if any
> of these operations are interrupted, the data can be completely recovered
> by leveldb repair.
>
> The only incomplete write that won't be recovered by the LevelDB repair
> process is the initial write to the recovery log, limiting exposure  to the
> key being actively written at the time of the snapshot/backup.  As long as
> 2 vnodes in the same preflist are not backed up while simultaneously
> writing the same key to the recovery log (i.e. rolling backups are good),
> this key will be recovered by AAE/read repair after restoration.
>

Joe,  thanks for the information.

Will Riak detect a LevelDB that was not cleanly shutdown when it starts up
and start repair on its own?

If not, will Basho add a command to riak-admin to make it easy to repair
LevelDB backends while restoring them?

Any comments on the backup strategy for Yokozuna?  Will it make use of
Solr's Replication Handler, or something more lower level?  Will the node
need to be offline to backup it up?

Ryan Zezeski any thoughts?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20140127/7332ef5e/attachment.html>


More information about the riak-users mailing list