Right, so you’ve decided to build an admission webhook. Congratulations, you’ve just volunteered to be the bouncer for the world’s most exclusive, and sometimes badly behaved, nightclub: your Kubernetes cluster. Your job is to look at every pod or deployment trying to get in and decide if it’s cool enough, or if you need to send it back to the curb to put on some proper labels (metaphorically speaking, of course).

Let’s get one thing straight from the start: an admission webhook is just an HTTP server that accepts a AdmissionReview request and returns a AdmissionReview response. The Kubernetes API server does all the heavy lifting of managing the webhook configuration; your job is to write the server that says “yes” or “no,” and maybe mutates the object on its way in. It’s deceptively simple, which is why most of the bugs are, frankly, our own fault.

The Absolute Basics: It’s Just JSON over HTTP

Don’t let the jargon intimidate you. At its core, the API server is just going to POST a JSON blob to an endpoint you specify. Your job is to:

  1. Parse that JSON.
  2. Decide if the request is allowed.
  3. Optionally, patch the object to mutate it.
  4. Send back a JSON blob with your decision.

The entire conversation is wrapped in the AdmissionReview object, which has the request in its request field and your response in its response field. You’ll be working with version admission.k8s.io/v1. Don’t use the older v1beta1; it’s deprecated and on its way out, and using it is like showing up to a black-tie event in flip-flops.

Your Webhook’s Two Hats: Validation and Mutation

You can write a webhook that validates (allows or denies), mutates (changes the object), or both. For the love of all that is holy, do not try to do both in a single webhook. It’s a classic “single responsibility principle” moment. The API server can call two different webhooks for the same request, so keep your validation logic and mutation logic separate. It makes your life, and your debugging, infinitely easier.

Writing a Validating Webhook in Python (with Flask)

Python is great for quickly banging out a logic-heavy webhook. Here’s a minimalist but complete example using Flask. We’re validating that every Pod must have an owner label, because chaos is not a valid management strategy.

from flask import Flask, request, jsonify
import base64
import json

app = Flask(__name__)

# This is the one and only endpoint that will receive requests
@app.route('/validate', methods=['POST'])
def validate():
    # The entire AdmissionReview request is in the JSON body
    admission_review = request.json
    admission_request = admission_review['request']

    # The object the user is trying to create/update
    pod_object = admission_request['object']
    labels = pod_object.get('metadata', {}).get('labels', {})

    # Our core business logic: check for the 'owner' label
    if 'owner' not in labels:
        allowed = False
        message = "Every pod must have an 'owner' label. This is for your own good."
    else:
        allowed = True
        message = "Pod has the required 'owner' label. Welcome to the cluster."

    # Build the response AdmissionReview object
    admission_response = {
        "uid": admission_request['uid'], # Critical: you MUST echo back the UID
        "allowed": allowed,
        "status": {"message": message}
    }

    response_admission_review = {
        "apiVersion": admission_review['apiVersion'],
        "kind": admission_review['kind'],
        "response": admission_response
    }

    return jsonify(response_admission_review)

if __name__ == '__main__':
    # Use TLS! The API server will only talk to you over HTTPS.
    app.run(ssl_context=('path/to/cert.pem', 'path/to/key.pem'), host='0.0.0.0', port=443)

The key here is the uid field. Forgetting to copy the UID from the request into the response is the number one cause of “my webhook just broke everything” panic. The API server uses it to match the response to the request. Fail this, and it fails hard.

Writing a Mutating Webhook in Go

Go is the lingua franca of Kubernetes, so it’s a natural fit. It’s more verbose, but you get type safety and performance. Here, we’ll mutate pods to inject a standard “environment” label if it’s missing.

package main

import (
	"encoding/json"
	"fmt"
	"net/http"

	admissionv1 "k8s.io/api/admission/v1"
	corev1 "k8s.io/api/core/v1"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/apimachinery/pkg/runtime"
	"k8s.io/apimachinery/pkg/runtime/serializer"
)

var (
	runtimeScheme = runtime.NewScheme()
	codecFactory  = serializer.NewCodecFactory(runtimeScheme)
	deserializer  = codecFactory.UniversalDeserializer()
)

func main() {
	http.HandleFunc("/mutate", mutateHandler)
	http.ListenAndServeTLS(":443", "cert.pem", "key.pem", nil)
}

func mutateHandler(w http.ResponseWriter, r *http.Request) {
	// Decode the AdmissionReview request
	var admissionReview admissionv1.AdmissionReview
	if err := json.NewDecoder(r.Body).Decode(&admissionReview); err != nil {
		http.Error(w, fmt.Sprintf("Error decoding JSON: %v", err), http.StatusBadRequest)
		return
	}

	// The juicy part of the request
	admissionRequest := admissionReview.Request
	var pod corev1.Pod
	if err := json.Unmarshal(admissionRequest.Object.Raw, &pod); err != nil {
		http.Error(w, fmt.Sprintf("Error unmarshaling pod: %v", err), http.StatusBadRequest)
		return
	}

	// Create a patch
	var patchOps []map[string]interface{}
	labels := pod.Labels
	if labels == nil {
		labels = make(map[string]string)
	}
	if _, exists := labels["environment"]; !exists {
		// Add the operation to create the /metadata/labels/environment path
		patchOps = append(patchOps, map[string]interface{}{
			"op":    "add",
			"path":  "/metadata/labels/environment",
			"value": "development", // You'd make this logic smarter, of course.
		})
	}

	// Marshal the patch into JSON bytes
	patchBytes, err := json.Marshal(patchOps)
	if err != nil {
		http.Error(w, fmt.Sprintf("Error marshaling patch: %v", err), http.StatusInternalServerError)
		return
	}

	// Build the response
	admissionResponse := &admissionv1.AdmissionResponse{
		UID:     admissionRequest.UID, // AGAIN, ECHO THE UID!
		Allowed: true,
		Patch:   patchBytes,
		PatchType: func() *admissionv1.PatchType {
			pt := admissionv1.PatchTypeJSONPatch
			return &pt
		}(),
	}

	responseAdmissionReview := admissionv1.AdmissionReview{
		TypeMeta: metav1.TypeMeta{
			APIVersion: admissionv1.SchemeGroupVersion.String(),
			Kind:       "AdmissionReview",
		},
		Response: admissionResponse,
	}

	w.Header().Set("Content-Type", "application/json")
	json.NewEncoder(w).Encode(responseAdmissionReview)
}

The crucial part here is the PatchType. You must set it to JSONPatch to tell the API server how to interpret the Patch field. This is a classic foot-gun; forget it, and your patch is silently ignored.

The Devil’s in the Details: Pitfalls and Paranoia

  • Idempotency is Non-Negotiable: Your mutating webhook might be called multiple times for the same request. If your patch isn’t idempotent, you’ll create a mess. The JSONPatch add operation to a path that already exists will fail. You must write your logic to check the current state before defining the patch.
  • Performance Matters: The API server is waiting on your response. Every millisecond you add is latency in kubectl commands. Make your logic fast and your startup faster. If your webhook is down, the API server might reject all requests depending on your failure policy. This is why you start with failurePolicy: Ignore.
  • TLS or Bust: You need a cert. The API server will not talk to you without one. Use a tool like cert-manager to manage this for you; rolling your own is a world of pain you don’t need.
  • Test Like Your Job Depends On It: Because it does. Test with curl by crafting your own AdmissionReview payloads. Test every edge case. What if the object is being deleted? What if fields are missing? Assume the user will send you the most malformed, bizarre request imaginable. Because they will.