Alright, let’s get our hands dirty with Calico. If you’ve been around the Kubernetes block, you’ve heard the name. It’s not just another CNI plugin; it’s more like the opinionated, highly-capable network engineer of the container world. While many CNIs slap a big virtual switch (VXLAN, I’m looking at you) over everything and call it a day, Calico often takes a different, radically sensible approach: it treats your data center like what it actually is—a network—and uses the Border Gateway Protocol (BGP) to have intelligent conversations with it.

Why BGP? Because it’s the language of the internet. It’s how massive networks run by different companies figure out how to talk to each other. Calico co-opts this battle-tested protocol to advertise the IP addresses of your pods to the rest of your network. No overlays, no encapsulation overhead (usually), just pure L3 routing. Your physical routers suddenly become aware that 10.244.103.12 is a real, routable address living on a specific worker node. It’s elegant, performant, and it makes a lot of network operators breathe a sigh of relief because they can actually see what’s going on.

How Calico Assigns IPs and Establishes Routes

Calico doesn’t use a central IPAM controller that could become a bottleneck. Instead, it gives each node its own block of IP addresses (a /26 by default, which is 64 IPs) from a larger pool you define. This is brilliantly simple. The kubelet on a node asks the local Calico CNI plugin for an IP when a pod starts, and the plugin just grabs the next one from its local block. No API call, no waiting.

The magic happens next. The calico-node pod running on each node, which houses the BGP speaker (a daemon called bird), then announces its block of IPs to the network. It does this either to your physical network gear (this is called “peer-to-peer” or “top-of-rack” mode) or to other Kubernetes nodes (this is the “node-to-node mesh” mode, which is the default for smaller clusters).

Here’s a quick look at the default IP pool, which you’ll almost certainly want to change to fit your network.

calicoctl get ippool default-ipv4-ippool -o yaml

apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
  name: default-ipv4-ippool
spec:
  blockSize: 26
  cidr: 192.168.0.0/16
  natOutgoing: true
  nodeSelector: all()
  vxlanMode: Never

See that cidr? That’s your pod network. blockSize: 26 defines the chunk each node gets. And natOutgoing: true is a crucial default: it means traffic leaving the cluster to the internet gets masqueraded (SNAT) to the node’s IP. You’ll likely turn this off in advanced setups where your network infrastructure can route the pod CIDR.

The Node-to-Node Mesh and When to Break It

The default node-to-node mesh mode is exactly what it sounds like: every node establishes a BGP session with every other node. It’s the Kubernetes equivalent of everyone in the office deciding they need to have a direct, one-on-one meeting with everyone else. It works great for clusters of, say, 50 nodes or fewer. Beyond that, the number of BGP sessions (n*(n-1)/2) becomes absurd and will start to overwhelm your nodes and your network controllers.

This is where you graduate to being a proper network admin. You disable the mesh and set up Calico to peer with your physical network routers (your “route reflectors”). This way, nodes only talk to a few central routers, which then propagate the routes intelligently. It’s a more scalable, sane architecture. Turning off the mesh is simple:

calicoctl create -f - <<EOF
apiVersion: projectcalico.org/v3
kind: BGPConfiguration
metadata:
  name: default
spec:
  nodeToNodeMeshEnabled: false
  asNumber: 64512
EOF

Now you’d configure your routers to peer with the nodes using AS number 64512 (a common private ASN).

The Crown Jewel: NetworkPolicy Enforcement

This is where Calico truly shines. Kubernetes NetworkPolicy is a great spec, but it’s just that—a spec. It’s up to the CNI to actually enforce it. Many do it with iptables, which is… fine. Calico does it with iptables too, but it does it well, and it also offers a mind-bendingly performant alternative called eBPF.

But more importantly, Calico provides its own CRDs that extend NetworkPolicy significantly. You get things like:

  • GlobalNetworkPolicy: Rules that apply to the whole cluster, perfect for “deny-all” defaults or mandatory security controls.
  • GlobalNetworkSet: A group of IPs (e.g., known malicious actors) you can reference in a policy for deny lists.
  • HostEndpoint: Allows you to manage policy for the node’s own interfaces, not just pods. This is huge for securing the nodes themselves.

Here’s a quick example of a default-deny-then-allow setup, which is security 101.

apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: default-deny-ingress
spec:
  order: 1000
  types:
  - Ingress
  selector: all()
  ingress:
  - action: Deny
---
apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: default-allow-egress
spec:
  order: 500
  types:
  - Egress
  selector: all()
  egress:
  - action: Allow

The order field is critical—policies are evaluated in order from lowest to highest. So we allow egress (order 500) but deny ingress (order 1000) by default.

Common Pitfalls and The VXLAN Fallback

First pitfall: assuming BGP will always work. Sometimes you’re in a cloud environment or have a network team that (understandably) doesn’t want a thousand Kubernetes nodes peering into their core routers. In this case, Calico can gracefully fall back to using VXLAN, an overlay network. It’s less performant than pure BGP, but it works and it’s still managed by Calico. You enable it by setting vxlanMode: Always in your IPPool.

Second pitfall: forgetting natOutgoing. If you disable this and your network infrastructure doesn’t have a route back to your pod CIDR, your pods can make calls out to the internet, but the replies will have nowhere to go. The packets vanish, and you spend an afternoon questioning your life choices.

Finally, the eBPF dataplane is fantastic, but it’s still maturing. Check the Calico docs for the latest supported features. Don’t just flip it on in production because it’s cool. But do test it, because when it’s supported for your use case, the performance benefits are very, very real.

Calico gives you the tools to build a robust, scalable, and incredibly secure network. It expects you to understand networking fundamentals, and in return, it doesn’t hide anything from you. It’s the CNI for grown-ups.