LevelDB driver

Justin Sheehy justin at basho.com
Mon Jul 4 10:14:14 EDT 2011


Hi, Jonathan.

On Mon, Jul 4, 2011 at 9:42 AM, Jonathan Langevin
<jlangevin at loomlearning.com> wrote:

> I've seen users show concern of Bitcask's space usage overhead. How does that
> compare against LevelDB?

Bitcask doesn't have much in the way of disk space "overhead" unless
you mean that the space used by deleted or overwritten values is not
reclaimed until after a merge. In that way LevelDB is similar since
space used by deleted and overwritten items is reclaimed as they are
moved into older "levels" of the DB. The behavior here is not
identical, but similar in concept.

By way of comparison, InnoDB imposes about a 2x space overhead cost on
many common datasets but the overhead is usually fairly static.

> If using a Level backend, what advantages do we lose of Bitcask? ls replication &
> availability an issue at all?

The functionality provided by Riak above the storage engines (such as
replication and system-wide availability) are generally not impacted
by your choice of storage engine.

There are two main things you would lose today:

1 - latency
2 - stability

The first of these is fundamental: for many usage patterns Bitcask
will have a latency advantage over LevelDB due to being able to
guarantee that it will never perform more than a single disk seek per
operation.

The second is just about the relative immaturity of LevelDB: we have
not yet seen LevelDB in production environments for an extended amount
of time as we have with Bitcask. Anyone using it now as a Bitcask
replacement should realize that they are on the leading edge and
taking the usual risks that come with adopting new software. That
said, we expect LevelDB to do well over time as one of the alternative
Riak storage engines.

The main reason to use LevelDB under Riak would be if your number of
keys is huge and thus the RAM consumption of Bitcask would make it
unsuitable. That is, we expect people to use LevelDB in the same
situations that they might previously have chosen Innostore as their
storage engine.

-Justin




More information about the riak-users mailing list