Maintenance

Right, let’s talk about the part everyone says is important but often tries to skip: making sure your precious applications don’t collectively faceplant when you flip the upgrade switch. Skipping this is like skydiving without checking your parachute because you “used it last week and it was fine.” Don’t be that person. The cluster’s new API server might be shinier, but your app is about to speak a language it no longer fully understands.

Look, I know you’re busy. The business is breathing down your neck for that new feature, and the idea of taking a perfectly good, running cluster and spending hours meticulously testing an upgrade feels like a luxury you can’t afford. I’m here to tell you it’s not a luxury; it’s your only life raft. Skipping this step is like skydiving and then checking if your parachute is in the bag. You will, at some point, have a catastrophic failure in production. The only question is whether you’ve practiced your emergency procedures in a safe environment first. A staging cluster is that safe environment. It’s where we break things on purpose so we don’t break them by accident later.

Alright, let’s get our hands dirty with the real first step of any upgrade: politely telling a node to stop accepting new work so we can kick it out for maintenance. This isn’t a suggestion; it’s a controlled, graceful shutdown of its workload. We do this with two simple but powerful concepts: cordon and drain. Think of it as the Kubernetes equivalent of putting an “Out for Lunch” sign on a door.

Right, so you’ve got a production cluster humming along nicely. It’s serving traffic, the metrics look good, and you’re feeling pretty smart. Then you remember: the Kubernetes version is about to fall out of support. Panic? No. We don’t do that. We do planned, methodical, slightly nerve-wracking upgrades. With managed services like EKS, GKE, and AKS, the cloud providers have done a lot of the heavy lifting, but they’ve also handed you a box of very powerful, very sharp tools. It’s your job not to accidentally amputate a production workload.

Right, so you’ve decided to upgrade your cluster with kubeadm. Good choice. It’s the officially blessed path, which means it’s less “holding a live badger” and more “performing delicate surgery.” Still, you’re holding a scalpel, not a chainsaw, so let’s proceed with a bit of finesse. The core idea here is that kubeadm upgrades one node at a time, and it does this by first upgrading the control plane nodes, one by one, and then the workers. You don’t just throw a switch and upgrade the whole thing at once; that’s a fantastic way to schedule an unplanned outage and order a pizza for a long, sad night.

Right, let’s talk about upgrade order. You’ve got your cluster, humming along nicely, and you’ve decided it’s time to drag it into the future. The single most important rule, the one you should tattoo on the inside of your eyelids, is this: you upgrade the control plane first, then the worker nodes. Always. This isn’t a suggestion; it’s the law of the land in Kubernetes. Break it, and you’re in for a world of hurt. I’ve seen people try to be clever and do it the other way around. They don’t do it twice.

Right, let’s talk about version skew. This isn’t some theoretical guideline dreamed up by a bored architect; it’s a hard-won set of rules that keeps your cluster from having a full-blown existential crisis during an upgrade. Think of it as the rules of engagement for a multi-version, distributed system. You can have different versions of components talking to each other, but only if they agree on the core rules of the conversation. Break those rules, and you get undefined behavior, which is a fancy term for “panic-induced outage at 2 AM.”