16.3 Flannel: Simple Overlay Network
Alright, let’s talk about Flannel. If Kubernetes networking were a high school clique, Flannel would be the reliable, unassuming friend who shows up with a six-pack and knows how to get the router working. It’s not the flashiest, it won’t brag about its features, but it will get the job done with a minimum of fuss. Its entire reason for existence is to solve one problem: making sure every Pod in your cluster gets a unique IP address that every other Pod (and Node) can talk to, regardless of which physical machine it’s sitting on. It does this using one of the oldest tricks in the networking book: an overlay network.
Think of an overlay like a private, encrypted VIP channel inside a crowded public subway car. Everyone else on the subway (the underlying network, or underlay) is just seeing generic encrypted traffic, but inside our channel, we’re having a coherent conversation about who owes whom IP addresses. Flannel creates this channel by encapsulating Pod network traffic inside another network packet (like putting a letter inside a larger envelope) and shipping it across your physical network to the correct destination node.
How Flannel Pulls This Off: The Magic of flanneld and CNI
When you install Flannel, you’re deploying a single binary agent called flanneld as a DaemonSet (one Pod per node, because of course it is). This little guy is the brains of the operation. Its first job upon waking up is to secure an IP address range, a subnet, for its host node from a central lease manager. In most setups, this manager is the Kubernetes API itself, using our old friend etcd as the backing store, though Flannel can also use the Kube API directly now, which is a much saner default.
Once a node has its own little /24 slice of the cluster’s overall CIDR (e.g., 10.244.3.0/24), the real work begins. The flanneld agent on each node programs the local operating system’s routing table. It essentially adds a rule that says: “Hey, if you see traffic destined for any other node’s Pod CIDR (like 10.244.5.0/24), don’t try to route it locally—just send it directly to the IP address of the node that owns that subnet.” This is the key. It turns a complex overlay problem into a simple routing problem for the kernel to handle.
But flanneld doesn’t handle the actual plumbing for individual Pods. That’s where the CNI part comes in. The CNI plugin—a tiny executable called flannel—is invoked by the kubelet whenever a Pod is created or destroyed. This plugin reads its configuration from a file that flanneld helpfully manages, /run/flannel/subnet.env, which tells it the node’s specific subnet and MTU.
# This is what /run/flannel/subnet.env typically looks like.
# The flannel CNI plugin uses these values to configure each pod's interface.
FLANNEL_NETWORK=10.244.0.0/16
FLANNEL_SUBNET=10.244.3.1/24
FLANNEL_MTU=1450
FLANNEL_IPMASQ=false
The CNI plugin then uses this info to create the Pod’s network namespace, plumb a veth pair, and assign the IP address from the node’s allocated subnet. It’s a beautiful separation of concerns: flanneld handles the cluster-wide network state, and the CNI plugin handles the local, per-Pod plumbing.
The Backends: Your Choice of Tunneling Protocol
Here’s where Flannel gets interesting. That “just send it to the other node” idea I mentioned earlier? It has multiple ways to actually do that, and you get to choose. This choice is the backend in Flannel’s configmap, and it’s a trade-off between simplicity, performance, and network environment.
VXLAN (The Default): This is the go-to. It encapsulates layer 2 Ethernet frames inside UDP packets. It’s widely supported and will work across most networks. The performance hit from the encapsulation/decapsulation is generally acceptable for most workloads. The
flanneldagent creates a virtual interface, usually calledflannel.1, to handle this. You can see your node’s VXLAN learning table to watch it map other node’s Pod networks to their node IPs:# Show the VXLAN forwarding database. This shows that traffic for 10.244.1.0/24 # should be encapsulated and sent to the node at 192.168.50.21. ip neigh show dev flannel.1host-gw (Host Gateway): This is the performance king, but it comes with a massive caveat. Instead of encapsulating packets, it just uses plain old IP routing. It sets up routes on the host that point directly to other nodes’ IPs. The catch? It requires all your cluster nodes to be on the same L2 network segment. If you have a router between your nodes,
host-gwis a non-starter because the router won’t have the routes Flannel creates. But if you can use it, the performance is nearly native.WireGuard: A newer, more secure option. It uses WireGuard for encryption and tunneling, which is fantastic for security and has surprisingly good performance for an encrypted tunnel. If you need encryption for your Pod traffic, this is a much better choice than the older, more cumbersome
ipsecbackend.
The Rough Edges and Pitfalls
Let’s be honest, no software is perfect.
The MTU Gotcha: This is the number one cause of mysterious “packets too big” errors. When you encapsulate a packet (e.g., with VXLAN), you’re adding new headers, which makes the overall packet bigger. If your underlying network has a lower MTU than your Pod network, these oversized packets will be dropped. Flannel tries to calculate this and sets the Pod MTU accordingly (hence the
1450value you often see, which accounts for the VXLAN overhead on a standard1500MTU network). But if your datacenter network uses jumbo frames or has an weird MTU, you must configure Flannel’smtusetting manually. Getting this wrong leads to horribly degraded performance.It’s Simple, Remember? Flannel’s strength is also its weakness. It provides basic L3 connectivity and not much else. Need advanced network policies to block traffic between pods? You’ll need to install Canal (Flannel + Calico’s policy engine) or a different CNI altogether. Need to integrate with external cloud load balancers? That’s mostly on the cloud provider’s CNI. Flannel gives you the basic highway system; you bring the traffic lights and toll booths.
So, when do you choose Flannel? When you need something dead simple that “just works” for a vanilla cluster on-premises or in a cloud that doesn’t have its own CNI (like AWS Elastic Kubernetes Service does). It’s the vanilla ice cream of CNI plugins: reliably good, a little boring, and the perfect foundation for more exciting toppings.