43.4 logrotate: Rotating, Compressing, and Pruning Log Files
Right, let’s talk about logrotate. You’re here because your disk space is screaming for mercy, or you’re just smart enough to know it will be soon. Log files are the digital equivalent of hoarding old newspapers; they just keep piling up until you can’t open the front door. logrotate is your friendly, automated cleanup crew. It’s not the flashiest tool, but it’s one of the most reliable workhorses in your sysadmin toolkit. It rotates, compresses, mails, and deletes log files according to rules you set. And it’s probably already installed on your system, silently doing its job for core services.
How It Works: The Basic Mechanics
logrotate is typically run as a daily cron job (/etc/cron.daily/logrotate). It doesn’t daemonize; it just wakes up once a day, checks its configuration, and does what needs doing. The magic is in its configuration files. The main one is /etc/logrotate.conf, but the real action is in the /etc/logrotate.d/ directory, where each service or application can have its own specific configuration snippet. This is brilliant because it means installing nginx can drop in its own nginx file in /etc/logrotate.d/ without you having to muck with a monolithic config.
The core concept is simple: when logrotate runs, it reads a log file defined in its config, and if it meets the criteria for rotation (e.g., it’s been a week, it’s over 100MB), logrotate will:
- Rename the current log file (e.g.,
app.logbecomesapp.log.1). - (Optionally) compress the old file (
app.log.1becomesapp.log.1.gz). - Tell the application to start writing to a fresh
app.log. - Eventually, prune logs older than a certain number of rotations.
Here’s a dead-simple example. Let’s say you have a custom application log at /var/log/my-app.log. A basic config for it would look like this:
/var/log/my-app.log {
daily
missingok
rotate 7
compress
delaycompress
notifempty
create 644 root root
}
Let’s break this down line-by-line, because that’s where the devil lives.
Dissecting a Configuration Block
daily: Rotate the log every day. Other options areweekly,monthly, orsize-based(e.g.,size 100M).missingok: Don’t throw a fatal error if the log file is missing. It’ll just move on. This is usually what you want.rotate 7: Keep 7 rotated logs before starting to delete the oldest ones. So you’ll havemy-app.log,my-app.log.1.gz,my-app.log.2.gz, … up tomy-app.log.7.gz.compress: Pretty straightforward. It usesgzipby default. You can change this withcompresscmdandcompressextif you’re feeling fancy and want to usebzip2orxz(for better compression, at the cost of CPU).delaycompress: This is a crucial one. It delays the compression of the previous log file by one rotation. So, after the first run, you’ll havemy-app.log(new),my-app.log.1(old, not compressed yet), andmy-app.log.2.gz. Why? Some daemons might still have a file handle open to the recently rotated log (my-app.log.1) for a brief moment. Trying to compress a file that’s still open for writing is… problematic. This gives the process a full rotation cycle to close and reopen its logs.notifempty: Don’t rotate the log if it’s empty. No point in cluttering your drive with zero-byte files.create 644 root root: After rotation, create a new empty log file with the specified permissions, owner, and group. This is vital. If your app runs as usermy-appbut the new log file is created asroot:root, your app might not be able to write to it anymore, and you’ll be staring at an empty log file wondering why your application exploded.
The Most Important Directive: postrotate
Here’s where most people get tripped up. How do you tell your application to stop writing to the old file and start writing to the new one? You politely ask it, using a postrotate script.
Most well-behaved applications respond to a SIGUSR1 signal to reopen their log files. Others might need a full restart. You define this inside the config block.
/var/log/nginx/*.log {
daily
missingok
rotate 52
compress
delaycompress
notifempty
create 644 nginx nginx
sharedscripts
postrotate
/usr/sbin/nginx -s reopen 2>/dev/null || true
endscript
}
The postrotate block is where you put the command to signal your process. For nginx, the -s reopen command is the right way. For other apps, it might be kill -USR1 $(cat /var/run/myapp.pid).
Notice sharedscripts? That tells logrotate to run the postrotate script only once after all the logs matching the pattern (e.g., both access.log and error.log) have been rotated, not once for each file. You almost always want this.
Testing Your Configuration (Don’t Skip This!)
The most common pitfall is writing a config, running logrotate, and then watching your application fail to log anything because the permissions are wrong or the signal command failed. Never let your logrotate cron job run a new config untested.
Test it. Force a run and check the output.
sudo logrotate -d /etc/logrotate.d/my-app-config
The -d flag is dry-run. It will show you what it would do, which is great for a first pass. But it won’t actually move files or run scripts. To do a real test, use the -v (verbose) and --force flags.
sudo logrotate -vf /etc/logrotate.d/my-app-config
Now go immediately check your log directory (ls -la /var/log/) and make sure the new log file was created with the right permissions and that your application is still happily writing to it. Trust me, this five-second check will save you from a world of debugging pain later. This is the difference between a pro and an amateur. Be a pro.