10.5 ConcurrencyPolicy: Allow, Forbid, Replace
Right, so you’ve got your Job set up to process a queue or crunch some numbers. It works. But what happens if the previous Job run hasn’t finished yet and the scheduled time for the next run rolls around? Chaos? A pile-up of angry, resource-hogging Pods? Kubernetes, thankfully, doesn’t just let this happen by default. It gives you a steering wheel called concurrencyPolicy to decide how to handle this exact scenario. This isn’t just a suggestion; it’s a critical piece of configuration for any non-trivial CronJob.
Think of it this way: your CronJob is set to run every minute, but the job itself takes 90 seconds to complete. You’re inherently faster at scheduling work than you are at finishing it. This is where the three policies come in, and your choice fundamentally changes the behavior of your system.
The Three Policies: Allow, Forbid, Replace
Let’s break down what each of these actually means in practice.
Allow is the “live dangerously” mode. If a scheduled run time occurs and a Job from the previous run is still active (i.e., its Pods are still running), the CronJob controller will just create another Job and its Pods. It doesn’t care. It follows the schedule with the blind devotion of a cuckoo clock.
You use this when absolute timing is more important than resource usage and you’re confident your infrastructure can handle the potential overlap. Maybe you’re monitoring a system and need a data point every minute, regardless of what happened with the last one. Use this sparingly, and only if you’ve sized your cluster to handle the potential pile-up.
apiVersion: batch/v1
kind: CronJob
metadata:
name: risky-monitor
spec:
schedule: "*/1 * * * *"
concurrencyPolicy: Allow # Let the chaos reign
jobTemplate:
spec:
template:
spec:
containers:
- name: monitor
image: busybox
command: ["sh", "-c", "sleep 90 && echo 'Took a long time...'"]
restartPolicy: OnFailure
Forbid is the sensible default, and frankly, the one you should start with. If it’s time for a new run but the old one is still kicking, the CronJob controller will just skip the new run. It logs a handy little error message (Job skipped because previous execution is still running or similar) and moves on with its life.
This is your go-to for anything that involves mutating state or accessing a shared resource where overlapping runs would be disastrous. It’s the polite policy. It waits for its turn.
apiVersion: batch/v1
kind: CronJob
metadata:
name: sensible-backup
spec:
schedule: "*/5 * * * *"
concurrencyPolicy: Forbid # The safe bet. No overlaps.
jobTemplate:
spec:
template:
spec:
containers:
- name: backup
image: postgres:15-alpine
command: ["pg_dump", "-h", "db-host", "mydb"]
restartPolicy: OnFailure
Replace is the weird one. On the surface, it sounds like a good compromise: instead of creating a new Job (like Allow) or doing nothing (like Forbid), it deletes the currently running Job and its Pods and then creates the new one.
Let’s be direct: this is a terrible idea for almost every use case. Why? Because it’s basically a SIGKILL. It doesn’t gracefully terminate your running job; it yanks the rug out from under it. If your job was in the middle of writing to a database, you’re left with a half-finished transaction. If it was uploading a file, you now have a partial file. It creates a high potential for data corruption and is almost never what you want. The only conceivable use case is for jobs that are truly idempotent and where the latest run is the only one that matters, but even then, letting the previous run finish is usually safer.
The Devil in the Details: startingDeadlineSeconds
Here’s a pro tip that often gets missed. concurrencyPolicy only matters if the CronJob controller is actually running and healthy to check the schedule. What if your control plane is down for a few minutes? You’ll miss runs, right?
Well, enter startingDeadlineSeconds. This field is a bit of a misnomer. It sets a time window (in seconds) within which a missed job run can still be started. If the controller comes back online and realizes it missed a scheduled time that fell within the last X seconds, it will trigger the job immediately.
But—and this is a big but—it also affects how concurrencyPolicy is enforced for these “catch-up” runs. If you have concurrencyPolicy: Forbid and multiple missed runs fall within the startingDeadlineSeconds window, the controller will only start the most recent missed run. It won’t queue them all up. This prevents a thundering herd of jobs all starting at once when the controller recovers, which is actually very smart design.