Riak on SAN

Victor Victor at boirefillergroup.com
Wed Oct 2 17:25:41 EDT 2013

I believe, even force removal of data directories, on one of the nodes of a
cluster, is not equivalent of removal those data from cluster, and Riak
should treat it like failure of storage on node. But you make me concerned
about security of a client side application, as running data removal command
on a cluster level, could be a real disaster.

And still, if I would want to keep offline  copy of data, it seems easier
for me to just replace every disk in data array with new drives, archive
them, state to system that they failed, and again hope for bitcask to
restore data, no downtime involved, and not any harder then SAN snapshot.
Sounds like win-win situation for me.


Victor P.


From: Alexander Sicular [mailto:siculars at gmail.com] 
Sent: Wednesday, October 02, 2013 4:12 PM
To: Victor
Cc: 'John E. Vincent'; 'riak-users'
Subject: Re: Riak on SAN


Cluster "redundancy" is no safeguard against the unknown. The only true
reliable protection is a complete offline backup in a separate facility not
run by the same provider as your primary facility. I'm not saying everyone
is running at that level of paranoia, but it is something to consider
against the value of your data.


What if you get rooted and someone runs something like for node in nodes rm
-rf myriakdata ?

-Alexander Sicular




On Oct 2, 2013, at 4:03 PM, "Victor" <Victor at boirefillergroup.com> wrote:

Excuse me, if I misunderstood something, but why you would even want to have
backup of a single node, if you are running 5 node cluster? Assuming your W
key value for put requests is higher then number of vnodes on each physical
node, scenario when you loose data because of single node failure does not
seems to be possible. And restoring failed node should not require data
backup, as backend hinted handoff should make all work for you and get
failed system back to state prior failure.  

Sure, backup of initial state would be helpful, as it would help to save
plenty of time on node re-setup, but redundancy on cluster-level seems
reliable enough.


From: riak-users [mailto:riak- <mailto:users-bounces at lists.basho.com>
users-bounces at lists.basho.com] On Behalf Of John E. Vincent
Sent: Wednesday, October 02, 2013 3:12 PM
To: riak-users
Subject: Re: Riak on SAN


I'm going to take a competing view here.


SAN is a bit overloaded of a term at this point. Nothing precludes a SAN
from being performant or having SSDs. Yes the cost is overkill for fiber but
iSCSI is much more realistic. Alternately you can even do ATAoE.


>From a hardware perspective, if I have 5 pizza boxes as riak nodes, I can
only fit so many disks in them. Meanwhile I can add another shelf to my SAN
and expand as needed. Additionally backup of a SAN is MUCH easier than
backup of a riak node itself. It's a snapshot and you're done. Mind you
nothing precludes you from doing LVM snapshots in the OS but you still need
to get the data OFF that system for it to be truly backed up.


I love riak and other distributed stores but backing them up is NOT a solved
problem. Walking all keys, coordinating the take down of all your nodes in a
given order or whatever your strategy is a serious pain point. 


Using a SAN or local disk also doesn't excuse you from watching I/O
performance. With a SAN I get multiple redundant paths to a block device and
I don't get that necessarily with local storage. 


Just my two bits.



On Wed, Oct 2, 2013 at 2:18 AM, Jeremiah Peschka <
<mailto:jeremiah.peschka at gmail.com> jeremiah.peschka at gmail.com> wrote:

Could you do it? Sure.

Should you do it? No.

An advantage of Riak is that you can avoid the cost of SAN storage by
getting duplication at the machine level rather than rely on your storage
vendor to provide it.

Running Riak on a SAN also exposes you to the SAN becoming your bottleneck;
you only have so many fiber/iSCSI ports and a fixed number of disks. The
risk of storage contention is high, too, so you can run into latency issues
that are difficult to diagnose without looking into both Riak as well as the
storage system.

Keeping cost in mind, too, SAN storage is about 10x the cost of consumer
grade SSDs. Not to mention feature licensing and support... The cost
comparison isn't favorable.

Please note: Even though your vendor calls it a SAN, that doesn't mean it's
a SAN.

On Oct 1, 2013 11:08 PM, "Guy Morton" < <mailto:Guy.Morton at bksv.com>
Guy.Morton at bksv.com> wrote:

Does this make sense?

Guy Morton
Web Development Manager
Brüel & Kjær EMS

This e-mail is confidential and may be read, copied and used only by the
intended recipient. If you have received it in error, please contact the
sender immediately by return e-mail. Please then delete the e-mail and do
not disclose its contents to any other person.

riak-users mailing list
 <mailto:riak-users at lists.basho.com> riak-users at lists.basho.com

riak-users mailing list
 <mailto:riak-users at lists.basho.com> riak-users at lists.basho.com


riak-users mailing list
 <mailto:riak-users at lists.basho.com> riak-users at lists.basho.com


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20131002/5e384ece/attachment.html>

More information about the riak-users mailing list