RiakCS poor s3 upload speeds 2MB/s
lbakken at basho.com
Wed Jan 21 11:20:09 EST 2015
You should be able to help with disk usage by "turning down" the
trigger and threshold values described here:
Your cluster will merge more data which should help with disk usage.
If your typical use is to create and delete objects frequently, this
lbakken at basho.com
On Wed, Jan 21, 2015 at 4:40 AM, Toby Corkindale <toby at dryft.net> wrote:
> On 21 January 2015 at 15:22, Luke Bakken <lbakken at basho.com> wrote:
>> Hi Toby -
>> Are you using the stock bitcask configuration for merging?
> Hi Luke,
> Yes, pretty much.
>> On Tue, Jan 20, 2015 at 5:07 PM, Toby Corkindale <toby at dryft.net> wrote:
>>> Hi Kota,
>>> I had a bit of an off-list chat about this a while ago, plus continued
>>> to investigate locally, and eventually achieved some faster speeds,
>>> around 15MByte/sec writes.
>>> Things that were changed:
>>> * Adjusted Riak CS GC to be spread out over the cluster much more.
>>> * Tweaked up the put buffers and concurrency further
>>> * Moved most of the files out of CS and into Amazon S3+Glacier
>>> * Switched from nginx to haproxy
>>> * simplified firewalling for internal clients
>>> Each one of those changes made a small to modest improvement, but
>>> overall combined to make a quite noticeable improvement.
>>> I did notice something odd though -- despite moving most of the data
>>> out of the cluster, the disk-space-in-use by Riak is still very large
>>> compared to the amount stored. I mean, we moved more than 90% of the
>>> data out of the cluster, yet the actual disk space used only halved.
>>> For every gigabyte of file stored in CS, dozens of gigabytes are
>>> actually on disk!
>>> Either the garbage collection algorithm is very, very lazy, or somehow
>>> something has gone a bit wrong in the past, which might have explained
>>> part of the performance problems.
>>> We're going to look at redeploying a new, fresh cluster based on Riak
>>> 2 in the not too distant future, once Riak CS looks like it's approved
>>> for use there, and maybe that'll clear all of this up.
More information about the riak-users