Right, let’s talk about CoreDNS. You’ve probably heard it’s the “DNS for Kubernetes,” which is true, but that undersells it. It’s less like a simple phone book and more like the hyper-intelligent, slightly overworked switchboard operator for your entire cluster. It’s the reason your my-app-service.default.svc.cluster.local doesn’t just resolve to a hopeful dream, but to an actual IP address that other pods can talk to.

Before we dive in, a moment of silence for its predecessor, kube-dns. It worked, mostly, but it was a bit of a Rube Goldberg device—a dnsmasq container, a kube-dns container, and a sidecar for metrics, all held together with hope and YAML. CoreDNS is a single, blessedly simple Go binary that does everything with far more grace and configurability. The Kubernetes designers made a great call ditching the franken-service.

How CoreDNS Actually Works (It’s Not Magic)

CoreDNS isn’t polling the Kubernetes API constantly. That would be inefficient and frankly, a bit rude. Instead, the Kubernetes API itself pushes service and endpoint information to CoreDNS. CoreDNS watches for changes to Services and EndpointSlices (the modern, more efficient replacement for Endpoints) and updates its in-memory database accordingly.

When your pod sends a DNS query to the CoreDNS Pod IP (which it gets from the kubelet at startup), CoreDNS checks its loaded zones. The key one for you is the cluster.local zone. This is where the magic happens. A query for nginx-service.default.svc.cluster.local gets broken down:

  • nginx-service: The Service name.
  • default: The namespace it lives in.
  • svc: This tells CoreDNS you’re looking for a Service (it can also handle Pod DNS records).
  • cluster.local: The base domain for the cluster.

CoreDNS matches this, finds the corresponding ClusterIP for that Service, and sends it back. Done. For a headless Service (one without a ClusterIP, defined with clusterIP: None), it returns the set of IPs of the individual Pods backing the service. This is incredibly useful for stateful applications that need direct peer discovery.

The All-Important Corefile

This is CoreDNS’s configuration file, its brain, its entire personality. It’s defined in a ConfigMap, because of course it is. Let’s see what the default one looks like. Go ahead, run this in your cluster:

kubectl get configmap -n kube-system coredns -o yaml

You’ll see something gloriously sensible. No XML, no JSON, just a clean, directive-based config called the Corefile.

.:53 {
    errors
    health {
        lameduck 5s
    }
    ready
    kubernetes cluster.local in-addr.arpa ip6.arpa {
        pods insecure
        fallthrough in-addr.arpa ip6.arpa
        ttl 30
    }
    prometheus :9153
    forward . /etc/resolv.conf
    cache 30
    loop
    reload
    loadbalance
}

Let’s break this down like a good joke:

  • errors: Logs… well, errors. Useful.
  • health: Provides a health endpoint on port 8080. If this fails, the pod is deemed unhealthy.
  • ready: An endpoint on port 8181 that returns 200 OK after CoreDNS is fully up and has loaded its plugins. This is crucial for startup sequencing.
  • kubernetes cluster.local ...: This is the big one. This plugin is what makes CoreDNS understand Kubernetes Services and Pods. The pods insecure option enables pod name-to-IP resolution (e.g., 1-2-3-4.default.pod.cluster.local), but it’s generally considered a security risk for production. The fallthrough directive tells CoreDNS to pass unmatched requests down the chain.
  • prometheus: Exposes metrics on port 9153. Feed this to your monitoring stack.
  • forward . /etc/resolv.conf: This is your escape hatch to the outside world. Any query that doesn’t match the cluster.local domain (like google.com) gets forwarded to the nameservers listed in the pod’s /etc/resolv.conf file, which are typically your node’s resolvers or something upstream.
  • cache: Reduces load by caching responses. The ttl in the kubernetes plugin helps manage this.
  • loop: Detects simple forwarding loops and nukes the process if it finds one (which sounds dramatic, but it’s better than a melting cluster).
  • reload: Allows automatic reload of the Corefile if the ConfigMap changes without restarting the pod. A fantastic quality-of-life feature.
  • loadbalance: Does a round-robin shuffle of identical A, AAAA, or MX records. This is why subsequent DNS queries for a service might return the IPs in a different order, providing a basic form of client-side load balancing.

When It Goes Sideways: Common Pitfalls

  1. The Most Common Offender: ndots. This is the culprit 90% of the time. Your pod’s /etc/resolv.conf has a options ndots:5 line. This means any DNS query with fewer than five dots in the name will first try appending all the search domains (default.svc.cluster.local, svc.cluster.local, cluster.local) before trying the absolute name. A query for redis inside a pod becomes redis.default.svc.cluster.local., which works. A query for my.database.com becomes my.database.com.default.svc.cluster.local. (which fails) before finally trying my.database.com. (which works). This generates massive, unnecessary load on CoreDNS. The fix? For external calls, use fully qualified domain names (end them with a dot), or, for heavy external callers, lower the ndots value in your pod spec.

    dnsConfig:
  options:
    - name: ndots
      value: "2"

  2. Pod Restart Loops: If your readinessProbe for CoreDNS is just a TCP check on port 53, it might start serving traffic before it’s actually ready to answer Kubernetes queries. This causes other pods that depend on service discovery to fail on startup, which… well, you see the loop. Always use the ready plugin’s endpoint (port 8181) for your readiness probe in your CoreDNS Deployment.

  3. Hitting the Default Limits: CoreDNS defaults to a relatively low cache size and can be memory constrained. In a large cluster with thousands of pods and services, you might see SERVFAIL responses under load. You need to tune this. You can increase cache size, add the autopath plugin for smarter ndots handling, or scale CoreDNS horizontally.