Multiple disks

Joseph Blomstedt Joseph.Blomstedt at gmail.com
Tue Mar 22 03:25:55 EDT 2011


Each vnode already opens a separate bitcask, therefore there isn't any
necessary factor preventing the desired behavior. It's just not coded
that way. While an individual bitcask must be a single directory,
there is no reason all vnodes need to open bitcasks within a shared
root directory.

Luckily, it's easy to change this behavior. In fact, I played around
with the idea awhile back. This question prompted me to find/release
the code:
https://github.com/jtuple/riak_kv/commit/a8ab33224651e6850aed385e4c05c1993916a3e5

That commit should apply against riak-0.14.1. It extends the bitcask
data_root config option to allow for multiple root paths as well as a
selection strategy (random or spread). Random just randomly chooses
one of the directories. Spread picks the directory containing the
fewest already-opened bitcasks -- although, this is a soft guarantee
since no effort is taken to address multiple vnodes choosing a
directory concurrently.

Using paths that correspond to different mounted drives should do the trick.

-Joe

riak opens a new bitcask for each vnode. Therefore, while bitcask
itself requires a single directory, forcing all vnodes to use the same
root directory is

On Mon, Mar 21, 2011 at 5:29 PM, Greg Nelson <grourk at dropcam.com> wrote:
> Hello,
> We are currently evaluating Riak for an application that will store large
> amounts of data in a write-heavy pattern.  We'd like to pack many disks into
> each machine.  Currently, it appears that Bitcask uses exactly one directory
> to store data.  What is the best way to have it use multiple disks?  Is this
> something Innostore would handle better?
> We'd like to avoid RAID since we'll be paying for redundancy at a higher
> level with Riak (N=3, etc.).
> We'd also like to avoid a JBOD type setup where a single disk failure brings
> the whole node down, as we'll obviously be increasing those odds with each
> disk.
> What I'm wondering is, can each node distribute its vnodes across many
> disks?  And if one of those disks fails, will Riak handle that appropriately
> (i.e., the other vnodes continue to operate normally and hand-off data when
> the new disk comes online)?
> Thanks!
> Greg
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>




More information about the riak-users mailing list