Riak 1.4.0: *Massive* disk space saving with eLevelDB

Jordan West jwest at basho.com
Fri Jul 26 11:16:01 EDT 2013


To clarify a bit further:

If you started with a fresh 1.4 cluster (or explicitly changed the
app.config setting) you are using a new on-disk format that applies to any
backend used by Riak, including LevelDB. There new format is more compact
but, like MvM said, the majority of savings here probably came from only
copying the most recent version of a key.

Cheers,
Jordan


On Fri, Jul 26, 2013 at 7:31 AM, Matthew Von-Maszewski
<matthewv at basho.com>wrote:

> Dave,
>
> Glad you are happy.
>
> The truth is that you gained space via the backup/restore process.  The
> data formats of 1.3.1 and 1.4 are the same.
>
> leveldb only removes dead / old key/values during its background
> compaction.  It could be days, even weeks in some cases, between when you
> write fresh key/values and when their old version are discarded.
>
> Your restore process only put the fresh keys back into the database.  The
> old versions were never part of the backup.
>
> Matthew
>
>
> On Jul 26, 2013, at 7:52 AM, Dave Brady <dbrady at weborama.com> wrote:
>
> Thank you Basho for the new on-disk format changes to eLevelDB!
>
> We have just migrating from 1.3.1 to 1.4.0, and since we wanted to change
> the ring size too, we used Dan Kerrigan's Data Migrator to backup/restore
> our buckets.
>
> The space savings are very impressive!  Each node went from using about
> 410 GB to about 250 GB.
>
> Terrific!
>
> --
> Dave Brady
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130726/5dcf9828/attachment.html>


More information about the riak-users mailing list