Alright, let’s talk about syslog. You’ve seen those cryptic messages scrolling through your system logs, right? They’re not just random text vomit; they’re actually structured messages following one of the oldest and most widely adopted protocols in computing. It’s the duct tape and baling wire of logging—it’s everywhere, it’s ugly, but it gets the job done. The protocol itself, defined in RFC 5424, is a standard for message logging that allows different devices and software to send event notification messages across an IP network. But we need to start with the classic, original format to understand its soul.

The Anatomy of a syslog Message

A traditional syslog message (the one defined in RFC 3164, which everyone uses even though it’s “obsolete”) has a specific structure. It’s not just a string of text; it’s a packet of information with defined parts. Let’s break one down:

<165>Oct 11 22:14:15 my-server su[12345]: pam_unix(su:session): session opened for user root by alice(uid=1000)

That first bit, <165>, is the Priority value. It’s a single number that encodes two crucial pieces of information: the Facility and the Severity. The rest is the message itself, containing a timestamp, a hostname, a process name (often with a PID in brackets), and finally, the actual event message.

Facilities: Who Am I?

The Facility answers the question, “What part of the system generated this message?” It’s a way to categorize logs by their source. The original designers had a very 1980s Unix view of the world, so the list is… charmingly dated.

There are 24 facilities (0-23), and the important ones are:

  • 0 (kern): Kernel messages. The OS itself is talking.
  • 1 (user): User-level messages. The default for many applications.
  • 3 (daemon): System daemons. Your sshds and nginxs live here.
  • 5 (auth): Security/authorization messages. Login attempts, sudo commands—pay attention to these.
  • 9 (cron): The cron daemon. Did your scheduled job run? Check here.
  • 16-23 (local0 - local7): These are your golden tickets. They’re reserved for custom use by your own applications. You should be using these instead of cluttering up ‘daemon’ or ‘user’.

Here’s the absurd part: Facility 2 was originally for the mail system, but it’s called mail, not smtp or pop3. Facility 4 was for the usenet news system. I’m not joking. It’s a beautiful relic. The protocol is so old it has a facility for a service most readers have never even heard of.

Severities: How Bad Is It?

The Severity answers the question, “How urgently should I look at this?” It’s a spectrum from “just a debug note” to “the entire data center is on fire.”

The severities, from most to least severe, are:

  • 0 (emerg): System is unusable. Panic condition. (Think: echo "0" > /proc/sysrq-trigger).
  • 1 (alert): Action must be taken immediately. (e.g., a primary database is down).
  • 2 (crit): Critical conditions. (e.g., a hardware failure).
  • 3 (err): Error conditions. Something failed, but the system isn’t dead.
  • 4 (warning): Warning conditions. Not an error, but you might want to check it out.
  • 5 (notice): Normal but significant condition. This is often where successful auth logs go, which is a questionable choice in my opinion. It pollutes the “normal” stream with security info.
  • 6 (info): Informational messages. Standard output for successful operations.
  • 7 (debug): Debug-level messages. The “noise floor” of logging.

The most common pitfall here is applications that get it wrong. I’ve seen countless programs log their standard startup sequence as warning because the developer was paranoid, rendering the severity level utterly useless. Always sanity-check how your key applications use these levels.

Encoding Facility and Severity into Priority

The magic number at the start of a message is calculated as: (Facility * 8) + Severity.

Let’s decode our example: <165>. 165 / 8 = 20.625. The facility is the integer part, 20. The severity is the remainder, 5.

  • Facility 20 is local4.
  • Severity 5 is notice.

So, this message is from an application we’ve configured to use local4, and it’s a notice-level event. This math is why facilities are 0-23 and severities are 0-7; it fits perfectly into a single byte.

Configuring rsyslog to Use Facilities

Let’s get practical. You have a custom application. You should not log it as daemon. You should use one of the local facilities. Here’s how you’d configure the gold standard, rsyslog, to handle it.

First, tell your application to log to local4. How you do this depends on the app. It might be a command-line flag (--syslog-facility local4) or an environment variable.

Next, create an rsyslog configuration file (/etc/rsyslog.d/10-my-app.conf) to route those messages to their own file.

# Capture any message with facility local4
local4.* /var/log/my-app.log

# If you want to split it by severity: local4.notice for one file, local4.err for another.
local4.notice /var/log/my-app-notice.log
local4.err /var/log/my-app-errors.log

Then tell rsyslog to reload its configuration.

sudo systemctl reload rsyslog

The Rough Edges and Best Practices

The classic BSD syslog protocol (RFC 3164) has some serious warts. The timestamp lacks a timezone and a year. No, really. This causes all sorts of fun parsing problems at the end of December. The newer RFC 5424 standard fixes this with a more structured, extensible format, but you’ll still see the old one everywhere.

Best practice? Use local0 through local7 for your own stuff. Be consistent in assigning them. Document which app uses which facility. And for the love of all that is holy, be thoughtful about severity levels. If it’s a normal, successful event, it’s info or notice, not warning. Save warning and err for things that are actually unexpected or problematic. Your future self, trying to find a real error in a sea of irrelevant “warnings,” will thank you.