We knew the answer to this question once, back when the world was young and full of truth. Without hesitation, we’d have spouted “Just take the output of lsof | wc -l!” And it’s true enough, in a general sort of way.
But if you asked me the same question now, my answer would be: “Why do you want to know?”
Are you, for instance, attempting to determine whether the number of open files is exceeding the limit set in the kernel? Then the output of lsof will be practically useless.
Let’s back up…
On *nix systems, there is a limit set in the kernel on how many open file descriptors are allowed on the system. This may be compiled in, or it may be tunable on the fly. In Linux, the value of this parameter can be read from and written to the proc filesystem.
[root@srv-4 proc]# cat /proc/sys/fs/file-max 52427 |
On this system 52,427 open file descriptors are permitted. We are unlikely to run out. If we wanted to change it, we’d do something like this:
[root@srv-4 proc]# echo "104854" > /proc/sys/fs/file-max [root@srv-4 proc]# cat /proc/sys/fs/file-max 104854 |
(Warning, change kernel parameters on the fly only with great caution and good cause)
But how do we know how many file descriptors are being used?
[root@srv-4 proc]# cat /proc/sys/fs/file-nr 3391 969 52427 | | | | | | | | maximum open file descriptors | total free allocated file descriptors total allocated file descriptors (the number of file descriptors allocated since boot) |
The number of open file descriptors is column 1 – column 2; 2325 in this case. (Note: we have read contradictory definitions of the second column in newsgroups. Some people say it is the number of used allocated file descriptors – just the opposite of what we’ve stated here. Luckily, we can prove that the second number is free descriptors. Just launch an xterm or any new process and watch the second number go down.)
What about the method we mentioned earlier, running lsof to get the number of open files?
[root@srv-4 proc]# lsof | wc -l 4140 |
Oh my. There is quite a discrepancy here. The lsof utility reports 4140 open files, while /proc/sys/fs/file-nr says, if our math is correct, 2325. So how many open files are there?
What is an open file?
Is an open file a file that is being used, or is it an open file descriptor? A file descriptor is a data structure used by a program to get a handle on a file, the most well know being 0,1,2 for standard in, standard out, and standard error.
The file-max kernel parameter refers to open file descriptors, and file-nr gives us the current number of open file descriptors. But lsof lists all open files, including files which are not using file descriptors – such as current working directories, memory mapped library files, and executable text files.
To illustrate, let’s examine the difference between the output of lsof for a given pid and the file descriptors listed for that pid in /proc.
Pick a PID, any PID
Let’s look at this process:
usr-4 2381 0.0 0.5 5168 2748 pts/14 S 14:42 0:01 vim openfiles.html [root@srv-4 usr-4]# lsof | grep 2381 vim 2381 usr-4 cwd DIR 3,8 4096 2621517 /n vim 2381 usr-4 rtd DIR 3,5 4096 2 / vim 2381 usr-4 txt REG 3,2 2160520 34239 /usr/bin/vim vim 2381 usr-4 mem REG 3,5 85420 144496 /lib/ld-2.2.5.so vim 2381 usr-4 mem REG 3,2 371 20974 /usr/lib/locale/LC_IDENTIFICATION vim 2381 usr-4 mem REG 3,2 20666 192622 /usr/lib/gconv/gconv-modules.cache vim 2381 usr-4 mem REG 3,2 29 20975 /usr/lib/locale/LC_MEASUREMENT vim 2381 usr-4 mem REG 3,2 65 20979 /usr/lib/locale/LC_TELEPHONE vim 2381 usr-4 mem REG 3,2 161 19742 /usr/lib/locale/LC_ADDRESS vim 2381 usr-4 mem REG 3,2 83 20977 /usr/lib/locale/LC_NAME vim 2381 usr-4 mem REG 3,2 40 20978 /usr/lib/locale/LC_PAPER vim 2381 usr-4 mem REG 3,2 58 51851 /usr/lib/locale/LC_MESSAGES/SYSL vim 2381 usr-4 mem REG 3,2 292 20976 /usr/lib/locale/LC_MONETARY vim 2381 usr-4 mem REG 3,2 22592 99819 /usr/lib/locale/LC_COLLATE vim 2381 usr-4 mem REG 3,2 2457 20980 /usr/lib/locale/LC_TIME vim 2381 usr-4 mem REG 3,2 60 35062 /usr/lib/locale/LC_NUMERIC vim 2381 usr-4 mem REG 3,2 290511 64237 /usr/lib/libncurses.so.5.2 vim 2381 usr-4 mem REG 3,2 24565 64273 /usr/lib/libgpm.so.1.18.0 vim 2381 usr-4 mem REG 3,5 11728 144511 /lib/libdl-2.2.5.so vim 2381 usr-4 mem REG 3,5 22645 144299 /lib/libcrypt-2.2.5.so vim 2381 usr-4 mem REG 3,5 10982 144339 /lib/libutil-2.2.5.so vim 2381 usr-4 mem REG 3,5 105945 144516 /lib/libpthread-0.9.so vim 2381 usr-4 mem REG 3,5 169581 144512 /lib/libm-2.2.5.so vim 2381 usr-4 mem REG 3,5 1344152 144297 /lib/libc-2.2.5.so vim 2381 usr-4 mem REG 3,2 173680 112269 /usr/lib/locale/LC_CTYPE vim 2381 usr-4 mem REG 3,5 42897 144321 /lib/libnss_files-2.2.5.so vim 2381 usr-4 0u CHR 136,14 16 /dev/pts/14 vim 2381 usr-4 1u CHR 136,14 16 /dev/pts/14 vim 2381 usr-4 2u CHR 136,14 16 /dev/pts/14 vim 2381 usr-4 4u REG 3,8 12288 2621444 /n/.openfiles.html.swp [root@srv-4 usr-4]# lsof | grep 2381 | wc -l 30 [root@srv-4 fd]# ls -l /proc/2381/fd/ total 0 lrwx------ 1 usr-4 usr-4 64 Jul 30 15:16 0 -> /dev/pts/14 lrwx------ 1 usr-4 usr-4 64 Jul 30 15:16 1 -> /dev/pts/14 lrwx------ 1 usr-4 usr-4 64 Jul 30 15:16 2 -> /dev/pts/14 lrwx------ 1 usr-4 usr-4 64 Jul 30 15:16 4 -> /n/.openfiles.html.swp |
Quite a difference. This process has only four open file descriptors, but there are thirty open files associated with it. Some of the open files which are not using file descriptors: library files, the program itself (executable text), and so on as listed above.
These files are accounted for elsewhere in the kernel data structures (cat /proc/PID/maps to see the libraries, for instance), but they are not using file descriptors and therefore do not exhaust the kernel’s file descriptor maximum.
> Have a look at the Latest Network Management & Monitoring Software for your Network Infrastructure
> Grab a Free Bandwidth Monitoring Tool and Keep an Eye on Bandwidth Hogs
> Download a FREE TFTP Server for Trivial File Transfers and More!
> Need to Monitor Netflow and Collector & Analyze Data? Here’s a List of Free Cisco Netflow Tools & Software!
> Wake on Lan is a Great Feature for waking Up Dormant PC’s – Here’s a List of Software & Tools to Assist in WOL Functions!
> IP Address Management & Tracking is a Great way to Keep Track of all IP Addresses in your Network!
> Use a Secure FTP Server for transferring (upload/download) – Free & Paid Versions available!
> Monitoring your Systems is Crucial – Find Paid & Free Tools & Software to do it Here!
> Analyzing and Collecting IPFix Flow Packets Is Easy with these Software Utilities!
and if you use sFlow, Use these Collectors & Analyzers for that Protocol!
> For Syslog Server Collection and Monitoring, Here’s the Top Tools/Software (Free!) to use!
> If you use Windows Management Instrumentation for Windows Systems, Here’s some WMI Monitors to use in your Infrastructure!
> Monitor your SNMP Devices, Servers and Network Equipment Quick & Ease,
and if you need a more Robust Infrastructure Monitoring Tool, grab one from HERE!
