Memory usage

Edgar Veiga edgarmveiga at gmail.com
Tue Feb 10 11:16:56 EST 2015


Hi all!

I have a riak cluster, working smoothly in production for about one
year, with the following characteristics:

- Version 1.4.12

- 6 nodes

- leveldb backend

- replication (n) = 3

~ 3 billion keys

~ 1.2Tb per node

- AAE disabled


Two days ago I've upgraded all of the 6 nodes from riak v1.4.8 to
v1.4.12, and two things started happening that are a little bit odd

1) The first is the memory consumption, please check the next imagem
to understand what I mean:

- https://dl.dropboxusercontent.com/u/1962284/riak.png

2) All of the machines keep logging hinted handoffs after the rolling
restart. I've made the upgrade on non-busy hours and assured that the
rolling restart was concluded only when all the in-progress handoffs
were concluded, but on the next day when checking the logs I've
realised that they keep appearing... Heres are some random examples:

2015-02-10 16:11:55.547 [info]
<0.3070.753>@riak_core_handoff_sender:start_fold:148 Starting
hinted_handoff transfer of riak_kv_vnode from 'riak at 192.168.20.112'
765004763290394496247241279624929393101152190464 to
'riak at 192.168.20.109' 765004763290394496247241279624929393101152190464

2015-02-10 16:11:55.548 [info]
<0.3070.753>@riak_core_handoff_sender:start_fold:236 hinted_handoff
transfer of riak_kv_vnode from 'riak at 192.168.20.112'
765004763290394496247241279624929393101152190464 to
'riak at 192.168.20.109' 765004763290394496247241279624929393101152190464
completed: sent 3.15 KB bytes in 1 of 1 objects in 0.00 seconds (3.99
MB/second)

2015-02-10 16:12:05.803 [info]
<0.3434.753>@riak_core_handoff_sender:start_fold:148 Starting
hinted_handoff transfer of riak_kv_vnode from 'riak at 192.168.20.112'
902020541790166644828836732692080926193895866368 to
'riak at 192.168.20.109' 902020541790166644828836732692080926193895866368

2015-02-10 16:12:05.856 [info]
<0.3368.753>@riak_core_handoff_sender:start_fold:148 Starting
hinted_handoff transfer of riak_kv_vnode from 'riak at 192.168.20.112'
570899077082383952423314387779798054553098649600 to
'riak at 192.168.20.111' 570899077082383952423314387779798054553098649600

2015-02-10 16:12:05.860 [info]
<0.3434.753>@riak_core_handoff_sender:start_fold:236 hinted_handoff
transfer of riak_kv_vnode from 'riak at 192.168.20.112'
902020541790166644828836732692080926193895866368 to
'riak at 192.168.20.109' 902020541790166644828836732692080926193895866368
completed: sent 39.79 KB bytes in 1 of 1 objects in 0.06 seconds
(699.32 KB/second)

2015-02-10 16:12:05.886 [info]
<0.3368.753>@riak_core_handoff_sender:start_fold:236 hinted_handoff
transfer of riak_kv_vnode from 'riak at 192.168.20.112'
570899077082383952423314387779798054553098649600 to
'riak at 192.168.20.111' 570899077082383952423314387779798054553098649600
completed: sent 3.55 KB bytes in 1 of 1 objects in 0.03 seconds
(118.58 KB/second)


Should I be worried or is this normal on this version?


Best regards,

Edgar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20150210/b7ae4655/attachment-0002.html>


More information about the riak-users mailing list