Riak KV performance in sensor data storage application

Martin Sumner martin.sumner at infinityworks.com
Tue Oct 3 13:22:02 EDT 2017


I suspect with the test environment having only 4GB of RAM and a single
slow disk per node - both Leveldb and Leveled are going to struggle in
terms of the write throughput.  As n=1 in this test, some of the advantages
of Leveled will be lost also (e.g. the n HEADs advantage).

I thought the paper was really interesting though.  It highlights an area
of riak performance - merge effectiveness - that is not often discussed.
I've ignored merge effectiveness as a measure between leveldb and leveled,
but it is an area where leveldb is currently notably better (and may well
be better than Bitcask).

As DeadZen mentioned having awareness of file system space is interesting.
The "fisherman's algorithm" approach mentioned in the paper is a neat, I've
not thought about having a "Target Usage Level" - and I think this is
potentially a better way of tuning a backend merge process.  I will try and
put some effort in writing up the approach to tuning merge effectiveness
that is implemented in leveled sometime soon for comparison.


On 3 October 2017 at 17:32, DeadZen <deadzen at deadzen.com> wrote:

> This looks cool, and might give some indication that back pressure
> should be aware of available file system space..
> Any plans on LevelDB testing? A newer option Leveled might be cool to
> compare with as well.
>
> On Tue, Oct 3, 2017 at 6:58 AM, Grigory Fateyev <gfborn at gmail.com> wrote:
> > Yury, thank you for sharing your investigations. It is very useful to
> read!
> >
> > 2017-10-03 8:32 GMT+03:00 Yury Shevchuk <sizif at botik.ru>:
> >>
> >> Greetings!
> >>
> >> We have published a paper on the subject.  It may be of interest for
> >> Riak developers and users who put Riak under heavy write load.
> >>
> >>   English: http://psta.psiras.ru/read/psta2017_3_61-85.pdf
> >>   Russian (native): http://psta.psiras.ru/read/psta2017_3_31-60.pdf
> >>
> >> Best,
> >>
> >>
> >> -- Yury
> >>
> >> _______________________________________________
> >> riak-users mailing list
> >> riak-users at lists.basho.com
> >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >
> >
> >
> > _______________________________________________
> > riak-users mailing list
> > riak-users at lists.basho.com
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>

-- 
***** Email confidentiality notice *****
This message is private and confidential.  If you have received this 
message in error, please let us know and remove it from your system.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20171003/5534b1ff/attachment-0002.html>


More information about the riak-users mailing list