File descriptor leaks?

Dmitry Demeshchuk demeshchuk at gmail.com
Thu Oct 14 07:48:59 EDT 2010


Greetings.

We have recently started to get the emfile errors. ulimit -n is 32767.
Restarting Riak helps for several hours and then we run out of
descriptors again.

Some time later after restart I performed lsof and found the following
descriptors:

kondemand   154       root  cwd   unknown
/proc/154/cwd (readlink: Permission denied)
kondemand   154       root  rtd   unknown
/proc/154/root (readlink: Permission denied)
kondemand   154       root  txt   unknown
/proc/154/exe (readlink: Permission denied)
kondemand   154       root NOFD
/proc/154/fd (opendir: Permission denied)
kondemand   155       root  cwd   unknown
/proc/155/cwd (readlink: Permission denied)
kondemand   154       root  cwd   unknown
/proc/154/cwd (readlink: Permission denied)
kondemand   154       root  rtd   unknown
/proc/154/root (readlink: Permission denied)
kondemand   154       root  txt   unknown
/proc/154/exe (readlink: Permission denied)
kondemand   154       root NOFD
/proc/154/fd (opendir: Permission denied)
kondemand   155       root  cwd   unknown
/proc/155/cwd (readlink: Permission denied)
kondemand   155       root  rtd   unknown
/proc/155/root (readlink: Permission denied)
kondemand   155       root  txt   unknown
/proc/155/exe (readlink: Permission denied)
kondemand   155       root NOFD
/proc/155/fd (opendir: Permission denied)
kondemand   156       root  cwd   unknown
/proc/156/cwd (readlink: Permission denied)
kondemand   156       root  rtd   unknown
/proc/156/root (readlink: Permission denied)
kondemand   156       root  txt   unknown
/proc/156/exe (readlink: Permission denied)
kondemand   156       root NOFD
/proc/156/fd (opendir: Permission denied)
kondemand   157       root  cwd   unknown
/proc/157/cwd (readlink: Permission denied)
kondemand   157       root  rtd   unknown
/proc/157/root (readlink: Permission denied)
kondemand   157       root  txt   unknown
/proc/157/exe (readlink: Permission denied)
kondemand   157       root NOFD
/proc/157/fd (opendir: Permission denied)
kondemand   158       root  cwd   unknown
/proc/158/cwd (readlink: Permission denied)
kondemand   158       root  rtd   unknown
/proc/158/root (readlink: Permission denied)
kondemand   158       root  txt   unknown
/proc/158/exe (readlink: Permission denied)

Also, the following couple of descriptors is opened several times at
the same time:

bash      20176        dem  mem       REG     252,0   256316   1179925
/usr/lib/locale/en_US.utf8/LC_CTYPE
bash      20176        dem  mem       REG     252,0       54   1179926
/usr/lib/locale/en_US.utf8/LC_NUMERIC
bash      20176        dem  mem       REG     252,0     2454   1179927
/usr/lib/locale/en_US.utf8/LC_TIME
bash      20176        dem  mem       REG     252,0   966938   1179928
/usr/lib/locale/en_US.utf8/LC_COLLATE
bash      20176        dem  mem       REG     252,0      286   1179929
/usr/lib/locale/en_US.utf8/LC_MONETARY
bash      20176        dem  mem       REG     252,0       52   1179930
/usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES
bash      20176        dem  mem       REG     252,0       34   1179931
/usr/lib/locale/en_US.utf8/LC_PAPER
bash      20176        dem  mem       REG     252,0       77   1179932
/usr/lib/locale/en_US.utf8/LC_NAME
bash      20176        dem  mem       REG     252,0      155   1179933
/usr/lib/locale/en_US.utf8/LC_ADDRESS
bash      20176        dem  mem       REG     252,0       59   1179934
/usr/lib/locale/en_US.utf8/LC_TELEPHONE
bash      20176        dem  mem       REG     252,0       23   1179935
/usr/lib/locale/en_US.utf8/LC_MEASUREMENT
bash      20176        dem  mem       REG     252,0    26048    917676
/usr/lib/gconv/gconv-modules.cache
bash      20176        dem  mem       REG     252,0      373   1179936
/usr/lib/locale/en_US.utf8/LC_IDENTIFICATION

Version of Riak is 0.12.1. There was a similar problem once and the
user was advised to make sure to use 0.12.1

Any ideas?

-- 
Best regards,
Dmitry Demeshchuk



More information about the riak-users mailing list