Right, so your primary server has decided to take an unscheduled vacation. It’s crashed, it’s gone, it’s pining for the fjords. The show must go on, and that means you need to promote one of your standby servers to take its place. This isn’t just a configuration change; it’s a state change. You’re telling a replica, “Stop following orders and start giving them.” It’s a big moment, and doing it correctly is the difference between a smooth transition and a full-blown, why-is-the-database-down-on-a-Tuesday disaster.

The Two Promotion Paths: Planned and Unplanned

You’ve got two main scenarios here, and your approach changes based on which one you’re in.

In a planned promotion, the primary is still up and you’re doing this for maintenance or a switchover. Here, you have the luxury of ensuring every last byte of data is flushed from the primary to the standby before you promote. This is the graceful waltz of pg_ctl promote or promote_trigger_file.

The unplanned promotion is the chaotic scramble. The primary is already dead, Jim. Your goal is to get a standby promoted with the most recent data it received, accepting that a tiny bit of data might be lost if the primary crashed before shipping its last transaction. This is where understanding WAL and replication slots becomes critical.

The promote_trigger_file Method (The Classic)

This is the old reliable method. You create a specific, empty file on the standby server’s filesystem. The standby’s postgres process is constantly checking for this file’s existence. When it finds it, it knows it’s showtime.

First, on the current primary (if it’s still alive!), you force it to flush all outstanding WAL. This is the polite thing to do.

-- On the current primary, if it's accessible
SELECT pg_switch_wal();

Now, on the standby server you want to promote, you create the magic file. The path is defined by the trigger_file setting in your recovery.conf (PostgreSQL 11 and earlier) or postgresql.conf (12 and later). Let’s say it’s /tmp/promote_me.

# On the standby server
touch /tmp/promote_me

That’s it. The server will detect the file, archive it (renaming it to avoid accidental re-promotion), and complete its recovery process. Check the logs; you’ll see glorious messages like “trigger file found” and “database system is ready to accept connections”.

Using pg_ctl promote (The Direct Approach)

PostgreSQL 9.3+ gave us a more direct command. This is often simpler because you don’t have to mess with file paths. Just run this on the standby server:

pg_ctl -D /your/data/directory promote

Under the hood, pg_ctl does the exact same thing as the trigger file—it creates a file named promote inside the data directory, which the server is also watching for. It’s just a cleaner interface. Use this if you can.

What Actually Happens During Promotion

This isn’t just a flag flip. The server goes through a precise sequence:

  1. Terminates Recovery: It stops replaying WAL records.
  2. Creates a Timeline: This is the brilliant part. It creates a new timeline with a new ID (e.g., if it was on timeline 1, it becomes timeline 2). This is a fork in the history of your database. If the old primary comes back online, its timeline (1) is now behind the new primary’s (2). This prevents it from accidentally sending old, conflicting data and corrupting everything. It’s PostgreSQL’s way of avoiding a time-travel paradox.
  3. Writes a End-of-Recovery (EOR) Record: It writes a special WAL record marking the exact point of promotion. This is the new beginning.
  4. Starts Normal Operation: It finally opens up for full read-write transactions.

The Aftermath: Dealing with the Old Primary

Here’s where everyone gets tripped up. Let’s say your old primary server reboots and comes back online. If you just start it, it will happily continue writing on its old timeline. It has no idea you’ve promoted a standby. You now have two primaries writing different data. This is a split-brain scenario, and it’s a database administrator’s worst nightmare.

You must never let this happen. The process is:

  1. Reclone the old primary: The safest bet. Wipe its data directory entirely and re-base it from the new primary using pg_basebackup. Treat the promoted standby as your new source of truth.
  2. Reconfigure it as a standby: If you’re absolutely sure no writes happened to it after it came back online (and you can’t ever be sure), you could theoretically edit its postgresql.conf to point to the new primary and hope for the best. But seriously, just reclone it. I’m not your boss, but I’m right about this.

Best Practices and Pitfalls

  • Test This: Practice promoting a standby in a staging environment. Know the steps by muscle memory. Panic is a terrible debugger.
  • Monitor Replication Lag: Promoting a standby that is gigabytes behind is pointless. You’ll lose data. Your monitoring should scream at you if lag exceeds a threshold.
  • recovery_target_timeline: If you ever need to fail back to an old primary (why?!), you can set this in its config to ’latest’ to make it follow the newest timeline. This is advanced surgery; handle with care.
  • Update Your Connection Strings: Your application’s connection pool needs to point to the new primary’s IP/hostname. This seems obvious, but you’d be surprised how often it’s forgotten in the panic. Use a load balancer or service discovery to make this dynamic.

Promotion is your emergency eject button. It’s not something you do for fun, but when you need it, you need it to work flawlessly. Understand the timelines, respect the split-brain danger, and always, always have a recent backup of your new primary before you do anything else to it.