speeding up bulk loading

Matthew Von-Maszewski matthewv at basho.com
Wed Aug 31 15:40:37 EDT 2016


Travis,

I cannot address the crash error you see in the logs.  Someone else will have to review that problem.

I want to point you to this wiki article:  https://github.com/basho/leveldb/wiki/riak-tuning-2

The article details how you can potentially increase Riak's throughput by restricting the number of schedulers (CPUs) assigned solely to Erlang.  Heavy bulk loading scenarios demonstrated 20 to 30% throughput gain.

Per the wiki article, you would create an (or add to your existing) advanced.config with the following based upon your 16 core machines:

[
    {vm_args, [{"+S", "14:14"}]}
].
Matthew
> On Aug 31, 2016, at 11:41 AM, Travis Kirstine <tkirstine at firstbasesolutions.com> wrote:
> 
> Magnus
>  
> Thanks for your reply.  We’re are using the riack C client library for riak (https://github.com/trifork/riack <https://github.com/trifork/riack>) which is used within an application called MapCache to store 256x256 px images with a corresponding key within riak.  Currently we have 75 million images to transfer from disk into riak which is being done concurrently.  Periodically this transfer process will crash
>  
> Riak is setup using n=3 on 5 nodes with a leveldb backend.  Each server has 45GB of memory and 16 cores with  standard hard drives.  We made no significant modification to the riak.conf except upping the leveldb.maximum_memory.percent to 70 and tweeking the sysctl.conf as follows
>  
> vm.swappiness = 0
> net.ipv4.tcp_max_syn_backlog = 40000
> net.core.somaxconn = 40000
> net.core.wmem_default = 8388608
> net.core.rmem_default = 8388608
> net.ipv4.tcp_sack = 1
> net.ipv4.tcp_window_scaling = 1
> net.ipv4.tcp_fin_timeout = 15
> net.ipv4.tcp_keepalive_intvl = 30
> net.ipv4.tcp_tw_reuse = 1
> net.ipv4.tcp_moderate_rcvbuf = 1
> # Increase the open file limit
> # fs.file-max = 65536 # current setting
>  
> I have seen this error in the logs
> 2016-08-30 22:26:07.180 [error] <0.20777.512> CRASH REPORT Process <0.20777.512> with 0 neighbours crashed with reason: no function clause matching webmachine_request:peer_from_peername({error,enotconn}, {webmachine_request,{wm_reqstate,#Port<0.2817336>,[],undefined,undefined,undefined,{wm_reqdata,'GET',...},...}}) line 150
>  
> Regards
>  
> From: Magnus Kessler [mailto:mkessler at basho.com <mailto:mkessler at basho.com>] 
> Sent: August-31-16 4:08 AM
> To: Travis Kirstine <tkirstine at firstbasesolutions.com <mailto:tkirstine at firstbasesolutions.com>>
> Cc: riak-users at lists.basho.com <mailto:riak-users at lists.basho.com>
> Subject: Re: speeding up bulk loading
>  
> On 26 August 2016 at 22:20, Travis Kirstine <tkirstine at firstbasesolutions.com <mailto:tkirstine at firstbasesolutions.com>> wrote:
> Is there any way to speed up bulk loading?  I wondering if I should be tweeking the erlang, aae or other config options?
>  
>  
>  
> Hi Travis,
>  
> Excuse the late reply; your message had been stuck in the moderation queue. Please consider subscribing to this list.
>  
> Without knowing more about how you perform bulk uploads, it's difficult to recommend any changes. Are you using the HTTP REST API or one of the client libraries, which use protocol buffers by default? What concerns do you have about the upload performance? Please let us know a bit more about your setup.
>  
> Kind Regards,
>  
> Magnus
>  
>  
> -- 
> Magnus Kessler
> Client Services Engineer
> Basho Technologies Limited
>  
> Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com <mailto:riak-users at lists.basho.com>
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20160831/a0c1a1a5/attachment-0002.html>


More information about the riak-users mailing list