Riak for streaming video

Alex De la rosa alex.rosa.box at gmail.com
Sun Jun 29 07:58:32 EDT 2014


Hi there,

One more question regarding Riak CS, does it behave like standard Riak?
storing 3 copies (default) and needing a minimum of 5 servers to operate?

Cheers,
Alex


On Fri, Jun 27, 2014 at 2:43 AM, Jared Morrow <jared at basho.com> wrote:

> Luwak is definitely EOL'd and RiakCS is our large object store going
> forward.  It is a far superior design compared to Luwak and handles very
> large file sizes.
>
>
>
> On Thursday, June 26, 2014, Jason Campbell <xiaclo at xiaclo.net> wrote:
>
>> Riak CS is designed to be the same interface as Amazon S3.  It can be run
>> anywhere you can run Riak.
>>
>> Think of it as a way to run your own S3.
>>
>> I think the hard limit on Riak objects is 50MB, but the recommended size
>> is less than 1MB per object.  The main reason for that is latency.  It's
>> faster to get 100x1MB chunks from the entire cluster than it is to get
>> 1x100MB chunk from a single server.  It's also easier for disks to pull a
>> 1MB object without delaying any other requests than pulling a 100MB object,
>> which may end up queuing other requests made after.
>>
>> There used to be a project called luwak that was built into Riak and
>> handled chunking files like this, but it has been EOL'd.  Perhaps someone
>> can shed some light on this.  Is RiakCS the final solution for binary
>> storage?  Or is there another way to store larger objects in Riak?
>>
>> RiakCS is wonderful, but it does seem overkill for some uses where you
>> don't need to expose an S3 interface, don't need the S3 permission
>> structure, and really just want file storage.
>>
>> ----- Original Message -----
>> From: "Alex De la rosa" <alex.rosa.box at gmail.com>
>> To: "Hector Castro" <hector at basho.com>
>> Cc: "riak-users" <riak-users at lists.basho.com>
>> Sent: Friday, 27 June, 2014 6:55:13 AM
>> Subject: Re: Riak for streaming video
>>
>>
>>
>> Hi Hector,
>>
>>
>> Thanks for your response :) highly appreciated. I have more questions
>> though.
>>
>>
>> Riak CS has to be implemented in Amazon S3? Or can I house it myself with
>> own managed servers?
>>
>>
>> What is the limit for an object in Standard Riak? Large objects should go
>> to Riak CS, but what would be a limit for the Standard Riak? I may want to
>> save pictures, for example, in binary data... not a RAW version, but like
>> processed photo uploads from users... I think Standard Riak can hold that
>> with no problem, but it would be good to know the size limit for a key's
>> content.
>>
>>
>> Thanks,
>> Alex
>>
>>
>>
>> On Thu, Jun 26, 2014 at 10:45 PM, Hector Castro < hector at basho.com >
>> wrote:
>>
>>
>> Hey Alex,
>>
>> My response are inline below.
>>
>> --
>> Hector
>>
>>
>>
>> On Thu, Jun 26, 2014 at 4:26 PM, Alex De la rosa
>> < alex.rosa.box at gmail.com > wrote:
>> > Hi Hector,
>> >
>> > I see, I always thought that Riak CS was the same as Riak just that you
>> run
>> > it in Amazon S3... they are actually different...
>>
>> Correct, but it is important to note that Riak is still at the core of
>> Riak CS.
>>
>>
>> > So... if I have a social network and one of the features is that they
>> can
>> > share video, I would use a normal Riak cluster for the webapp and data
>> and a
>> > Riak CS cluster for the video storage/streaming... am I right?
>>
>> That seems like a reasonable conclusion.
>>
>> Riak is fundamentally a distributed key/value store for low latency
>> access to smaller pieces of data. Riak CS is an S3/Swift
>> API-compatible object storage platform with a little higher request
>> latency, but also the added ability to house very large objects
>> (backups, raw images, video).
>>
>>
>> > By the way, couldn't this "range header" be implemented in standard
>> Riak?
>> > might be a good thing to have so you don't need 2 clusters for this
>> matter.
>>
>> It could some day. Right now, separating the clusters is best because
>> tuning a single cluster for both use cases would be difficult.
>>
>>
>>
>> > Thanks,
>> > Alex
>> >
>> >
>> > On Thu, Jun 26, 2014 at 9:28 PM, Hector Castro < hector at basho.com >
>> wrote:
>> >>
>> >> Hi Alex,
>> >>
>> >> For this type of problem, you may want to look into Riak CS. [0] It is
>> >> an object storage platform built on top of Riak.
>> >>
>> >> When you GET an object from Riak CS, you can specify a Range header to
>> >> get the object by its byte offset. [1]
>> >>
>> >> --
>> >> Hector
>> >>
>> >> [0] http://docs.basho.com/riakcs/latest/
>> >> [1]
>> >>
>> http://docs.basho.com/riakcs/latest/references/apis/storage/s3/RiakCS-GET-Object/#Examples
>> >>
>> >> On Thu, Jun 26, 2014 at 3:22 PM, Alex De la rosa
>> >> < alex.rosa.box at gmail.com > wrote:
>> >> > Hi there,
>> >> >
>> >> > Imagine that I want to use Riak for a video service like Youtube and
>> I
>> >> > save
>> >> > the video file in Riak... is there a way to stream the contents of 1
>> >> > key? I
>> >> > know we can stream keys... but what about its content?
>> >> >
>> >> > It would be pretty troublesome to have to wait for the full data to
>> be
>> >> > downloaded $bucket->get('myvideo') to be able to serve it... as they
>> can
>> >> > be
>> >> > pretty big files.
>> >> >
>> >> > Is there any recommendation to store/get big files like videos? or it
>> >> > would
>> >> > be better to use some other system than Riak for the job?
>> >> >
>> >> > Thanks,
>> >> > Alex
>> >> >
>> >> > _______________________________________________
>> >> > riak-users mailing list
>> >> > riak-users at lists.basho.com
>> >> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> >> >
>> >
>> >
>>
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20140629/7c02e9ef/attachment.html>


More information about the riak-users mailing list