Right, file descriptors. The humble, unassuming integer that the kernel hands out every time you open a file, a socket, or just about anything else. Think of them as tickets. The kernel is the bouncer at an exclusive club (your system resources), and every process needs a ticket to get in. Now, what happens when the bouncer runs out of tickets? Chaos. Connection refusals. Crashes. A logging daemon that suddenly can’t write to its log file. It’s a bad night.

This is why we have two distinct but deeply intertwined limits governing these tickets: the system-wide fs.file-max kernel parameter and the per-process nofile ulimit. Most of the confusion around this topic comes from not understanding their relationship. Let’s untangle it.

The Global Cap: fs.file-max

This is the big one. fs.file-max is the kernel’s hard limit on the total number of file descriptors that can be allocated across the entire system. It’s the total number of tickets the bouncer has in his entire roll. If this limit is hit, the kernel will simply refuse to hand out any more descriptors, full stop, regardless of how any individual process is configured.

You can check your current setting faster than you can mispronounce “sysctl”:

sysctl fs.file-max

On a modern system, you’ll probably see something like fs.file-max = 9223372036854775807. That’s not a typo; it’s 9.2 quintillion, which is the kernel’s way of saying “practically unlimited” on a 64-bit system. But on older or memory-constrained systems, you might see a much lower number based on your RAM.

If you need to raise it (because, say, you’re running a database server that insists on opening a new connection for every single query), you can do it temporarily:

# Elevate to root for this
sudo sysctl -w fs.file-max=1000000

Or, to make it stick after a reboot, add this line to /etc/sysctl.conf or, better yet, a new file in /etc/sysctl.d/ (because organization is a virtue):

# /etc/sysctl.d/99-file-descriptors.conf
fs.file-max = 1000000

Why would you need to change it? If you’re running a high-traffic web server, a large database, or anything that juggles thousands of network connections and files simultaneously. For a desktop or small server, the default is usually just fine.

The Per-Process Quota: nofile ulimit

Here’s where the real fun begins. The nofile ulimit is not a kernel parameter; it’s a per-process limit enforced by the shell (or init system). It dictates the maximum number of tickets a single process can ask the bouncer for.

There’s a soft limit (which you can change up to the hard limit) and a hard limit (the absolute ceiling for that process). Check your current session’s limits:

ulimit -Sn  # Shows the Soft limit
ulimit -Hn  # Shows the Hard limit

Now, the critical part everyone misses: A process’s nofile hard limit cannot exceed the system-wide fs.file-max. It’s a subset. If fs.file-max is 10,000, it doesn’t matter if you set a process’s hard limit to 20,000; it will still only be able to allocate up to 10,000. The bouncer only has 10,000 tickets total to give out.

The most common pitfall? A service like nginx or mysql hits its nofile limit long before the system gets anywhere near fs.file-max. You’ve given the bouncer a huge roll of tickets, but you told the nginx process it can only ask for 1024 of them. This is the classic “Too many open files” error.

To fix this, you need to raise the limit for the specific service. This is done not with sysctl but by configuring your init system. For systemd, you edit the service file or, preferably, use an override:

# Create an override directory for the nginx service
sudo systemctl edit nginx.service

This opens an editor. Add these lines to override the default limits:

[Service]
LimitNOFILE=65535

Save and exit, then reload systemd and restart the service:

sudo systemctl daemon-reload
sudo systemctl restart nginx.service

For processes started by a shell, you can set it in /etc/security/limits.conf. This file is a bit finicky and famously doesn’t work for processes started by systemd (which uses its own system), but for user login sessions, it’s the place:

# /etc/security/limits.conf - Example line
myappuser   hard    nofile    50000

Best Practices and The Gotcha

First, always check both limits when troubleshooting. Use cat /proc/sys/fs/file-max for the system cap and cat /proc/<PID>/limits for a running process’s actual limits.

The biggest gotcha? The difference between the number of open files and the maximum file descriptor number. The kernel’s internal arrays are sparsely populated. If a process opens and closes files wildly, it can end up with a very high-numbered file descriptor (like fd 500,000) even though it only has one file open. This can cause it to bump into its nofile limit prematurely. It’s a weird implementation detail, but it happens.

So, the best practice is this: set fs.file-max to a comfortably high value for your entire system. Then, be surgical and raise the nofile hard limit only for the specific processes that actually need it (your web server, your database). There’s no prize for giving your cron jobs a limit of 100,000. This gives you both the capacity you need and a measure of protection against a rogue process trying to grab all the tickets and shut down the whole club.