Riak KV performance in sensor data storage application

Yury Shevchuk sizif at botik.ru
Thu Oct 5 07:44:46 EDT 2017


Greetings!

Thanks to everybody who cared to read the PSTA article.

DeadZen: we have no plans for LevelDB/LevelEd experiments for now.
We seem to have Bitcask doing what we need at the moment and switched
to application development.  Thanks for bringing LevelEd to our
attention, it looks promising for the future.

Martin: I have discovered the LevelEd treatise at
https://github.com/martinsumner/leveled/blob/master/docs/
and enjoy reading it.  Thank you!

On Tue, Oct 03, 2017 at 06:22:02PM +0100, Martin Sumner wrote:
> I suspect with the test environment having only 4GB of RAM and a single
> slow disk per node - both Leveldb and Leveled are going to struggle in
> terms of the write throughput.  As n=1 in this test, some of the advantages
> of Leveled will be lost also (e.g. the n HEADs advantage).
> 
> I thought the paper was really interesting though.  It highlights an area
> of riak performance - merge effectiveness - that is not often discussed.
> I've ignored merge effectiveness as a measure between leveldb and leveled,
> but it is an area where leveldb is currently notably better (and may well
> be better than Bitcask).
> 
> As DeadZen mentioned having awareness of file system space is interesting.
> The "fisherman's algorithm" approach mentioned in the paper is a neat, I've
> not thought about having a "Target Usage Level" - and I think this is
> potentially a better way of tuning a backend merge process.  I will try and
> put some effort in writing up the approach to tuning merge effectiveness
> that is implemented in leveled sometime soon for comparison.
> 
> 
> On 3 October 2017 at 17:32, DeadZen <deadzen at deadzen.com> wrote:
> 
> > This looks cool, and might give some indication that back pressure
> > should be aware of available file system space..
> > Any plans on LevelDB testing? A newer option Leveled might be cool to
> > compare with as well.
> >
> > On Tue, Oct 3, 2017 at 6:58 AM, Grigory Fateyev <gfborn at gmail.com> wrote:
> > > Yury, thank you for sharing your investigations. It is very useful to
> > read!
> > >
> > > 2017-10-03 8:32 GMT+03:00 Yury Shevchuk <sizif at botik.ru>:
> > >>
> > >> Greetings!
> > >>
> > >> We have published a paper on the subject.  It may be of interest for
> > >> Riak developers and users who put Riak under heavy write load.
> > >>
> > >>   English: http://psta.psiras.ru/read/psta2017_3_61-85.pdf
> > >>   Russian (native): http://psta.psiras.ru/read/psta2017_3_31-60.pdf
> > >>
> > >> Best,
> > >>
> > >>
> > >> -- Yury




More information about the riak-users mailing list