Right, so you’ve configured your webhook, pointed it at your brilliant custom admission controller, and are now greeted by a cryptic error in your Kubernetes API server logs. My money’s on a TLS handshake failure. Why? Because the API server is, rightly, paranoid about who it talks to. It won’t just send your sensitive admission review requests to any old HTTP endpoint that asks; it needs to verify the server’s identity via TLS. And it has some very specific, and frankly, fussy requirements about how that server presents its certificate.

Let’s cut through the noise. The API server, acting as a client, needs to trust the certificate presented by your webhook server. This boils down to three things: the certificate must be valid, it must be signed by a Certificate Authority (CA) the API server trusts, and its Subject Alternative Name (SAN) must match the hostname in the webhook’s clientConfig.service or the URL you provided.

The Certificate and The CA Bundle

You have two players here:

  1. The server certificate: What your webhook service presents.
  2. The CA bundle: What the API server uses to verify that server certificate.

The most common, and often most robust, approach is to use Kubernetes-native tools. You’ll create a secret holding your CA’s public certificate and then inject it into the caBundle field of your ValidatingWebhookConfiguration or MutatingWebhookConfiguration. The API server will use this bundle to verify your webhook’s cert.

Here’s how you’d do it with cert-manager, the tool that saves us all from manual openssl incantations and the carpal tunnel that comes with them. First, you define an Issuer (or ClusterIssuer) for your webhook to get certificates from.

apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: my-webhook-issuer
  namespace: my-webhook-namespace
spec:
  selfSigned: {}

Then, you create a Certificate resource. This is where the magic happens—you define the details of the cert you need. Pay special attention to the dnsNames; this is the SAN field and it must match your webhook service’s DNS name.

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: my-webhook-cert
  namespace: my-webhook-namespace
spec:
  secretName: my-webhook-tls-secret # This is where the cert & key will be stored
  issuerRef:
    name: my-webhook-issuer
    kind: Issuer
  commonName: my-webhook-svc.my-webhook-namespace.svc # Often works, but SAN is king.
  dnsNames:
  - my-webhook-svc.my-webhook-namespace.svc # The fully qualified service name
  - my-webhook-svc.my-webhook-namespace.svc.cluster.local # Sometimes needed for FQDN

Now, cert-manager will generate a CA and a signed certificate, plopping them into the secret my-webhook-tls-secret. You mount this secret to your webhook’s Pod. Finally, you reference the CA certificate from that same secret in your webhook configuration. You can extract it with a command like this and patch your webhook:

# Get the CA cert from the secret, base64 decode it, then base64 encode it again for the YAML field (because Kubernetes loves base64).
ca_bundle=$(kubectl get secret my-webhook-tls-secret -n my-webhook-namespace -o jsonpath='{.data.ca\.crt}')

Then, in your webhook configuration YAML:

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  name: "my-webhook.example.com"
webhooks:
- name: "my-webhook.example.com"
  clientConfig:
    service:
      name: my-webhook-svc
      namespace: my-webhook-namespace
      path: "/validate"
    caBundle: {{ ca_bundle }} # This value gets inserted here

The Absolutely Critical SAN Field

This is the part that gets everyone. The commonName field is basically legacy at this point. Modern TLS validation, including what the Kubernetes API server uses, completely ignores it and relies solely on the Subject Alternative Name (SAN). If your certificate has a commonName of webhook.example.com but no SAN entry for it, it will be rejected. If your webhook’s URL is my-webhook-svc.my-webhook-namespace.svc, your certificate must have that exact string in its dnsNames list. Not “my-webhook-svc”, not “my-webhook-svc.my-webhook-namespace”. The full monty. I’ve lost hours of my life to this particular bit of pedantry.

When You Don’t Control the API Server’s Trust Store

Maybe you’re using a public CA like Let’s Encrypt, or your company’s internal PKI. In this case, your webhook’s certificate is signed by a CA that is already trusted by the API server (e.g., its root certificates are in the API server’s trust store). This is the simplest scenario—you can often leave the caBundle field in your webhook configuration completely empty. The API server will verify your cert against its own system-wide trust store. The only hurdle is ensuring your certificate’s SAN matches the service name.

The Common Pitfalls (AKA How I Learned to Stop Worrying and Love TLS)

  1. Wrong SAN: I’m saying it again because it’s that important. Your certificate’s SAN must match the webhook URL exactly. Use openssl s_client -connect my-webhook-svc.my-webhook-namespace.svc:443 -showcerts </dev/null 2>/dev/null | openssl x509 -noout -text | grep DNS: to see what your server is actually presenting.
  2. Expired Certificates: This seems obvious until your cluster has been running for a year and suddenly all your webhooks break. Cert-manager is your friend here for auto-renewal.
  3. Serving on the Wrong Port: Your webhook service must be serving HTTPS on port 443. The API server will not call it on 8443, 9443, or any other “logical” port you choose. It uses the standard port for the service.
  4. Skipping the caBundle: Unless you’re using a universally trusted CA, you must provide the caBundle. If you’re using a private CA and omit this, the API server has no way to trust your webhook’s certificate.

The TLS handshake is the bouncer at the club door. If your certificate doesn’t have the right credentials (a valid signature from a trusted CA and the correct name on the list), your webhook isn’t getting in. Get this right, and you can move on to the actually interesting part: writing the logic that decides which pods get to run.