Riak 1.4.0: *Massive* disk space saving with eLevelDB

Pedram Nimreezi mc at majorcomputing.com
Fri Jul 26 13:32:12 EDT 2013


A vacuum command would be most appropriate...


On Fri, Jul 26, 2013 at 11:16 AM, Jordan West <jwest at basho.com> wrote:

> To clarify a bit further:
>
> If you started with a fresh 1.4 cluster (or explicitly changed the
> app.config setting) you are using a new on-disk format that applies to any
> backend used by Riak, including LevelDB. There new format is more compact
> but, like MvM said, the majority of savings here probably came from only
> copying the most recent version of a key.
>
> Cheers,
> Jordan
>
>
> On Fri, Jul 26, 2013 at 7:31 AM, Matthew Von-Maszewski <matthewv at basho.com
> > wrote:
>
>> Dave,
>>
>> Glad you are happy.
>>
>> The truth is that you gained space via the backup/restore process.  The
>> data formats of 1.3.1 and 1.4 are the same.
>>
>> leveldb only removes dead / old key/values during its background
>> compaction.  It could be days, even weeks in some cases, between when you
>> write fresh key/values and when their old version are discarded.
>>
>> Your restore process only put the fresh keys back into the database.  The
>> old versions were never part of the backup.
>>
>> Matthew
>>
>>
>> On Jul 26, 2013, at 7:52 AM, Dave Brady <dbrady at weborama.com> wrote:
>>
>> Thank you Basho for the new on-disk format changes to eLevelDB!
>>
>> We have just migrating from 1.3.1 to 1.4.0, and since we wanted to change
>> the ring size too, we used Dan Kerrigan's Data Migrator to backup/restore
>> our buckets.
>>
>> The space savings are very impressive!  Each node went from using about
>> 410 GB to about 250 GB.
>>
>> Terrific!
>>
>> --
>> Dave Brady
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>


-- 
/* Sincerely
--------------------------------------------------------------
Pedram Nimreezi - Chief Technology Officer  */

// The hardest part of design … is keeping features out. - Donald Norman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130726/7cd325c6/attachment.html>


More information about the riak-users mailing list