25.1 Custom Resource Definitions (CRDs): Extending the Kubernetes API
Right, so you’ve got Kubernetes. It’s brilliant. It knows how to run Deployments, Services, Pods—the whole gang. But what if you need it to know about something else? Something specific to your company’s bizarrely complex and beautiful internal architecture? You don’t just want to run a pod; you want to tell Kubernetes, “Hey, I need a DatabaseCluster with 3 replicas, encrypted storage, and a weekly backup to this S3 bucket.”
You can’t just yell that at the API server. It would stare back at you blankly, like a dog being shown a card trick. This is where Custom Resource Definitions (CRDs) come in. Think of a CRD as your way of walking up to the Kubernetes API and saying, “We’re adding a new noun to our vocabulary. Learn it.” You’re formally extending the Kubernetes API to understand your new custom resource (CR).
Let’s be clear: the CRD defines the new resource type. The actual instances you create are called custom resources. It’s the difference between defining a class in code (the CRD) and creating an object of that class (the custom resource).
The Anatomy of a CRD
At its heart, a CRD is a YAML file that describes the shape of your new resource. The key parts are the group, version, kind (collectively known as a GVK), and the schema. The schema, defined using OpenAPI v3, is the most important part. It’s how you tell Kubernetes what fields your custom resource has, what types they are, and whether they’re required. Without a schema, you’ve just created a fancy, but dangerously unstructured, key-value store. Don’t do that.
Here’s a simple example for a CronTab resource (a classic example, and yes, it’s as silly as it sounds). We’re putting it in the stable.example.com API group.
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
# Name must be plural.<group>
name: crontabs.stable.example.com
spec:
group: stable.example.com
versions:
- name: v1
served: true # Make this version available via the API
storage: true # Use this version for storing objects
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
cronSpec:
type: string
image:
type: string
replicas:
type: integer
required: ["cronSpec", "image"] # Force the user to provide these
scope: Namespaced # Or Cluster for a cluster-scoped resource
names:
# kind is the CamelCased singular type.
kind: CronTab
# plural is the plural name to use in the URL (/apis/<group>/<version>/<plural>)
plural: crontabs
# singular is for display in CLI.
singular: crontab
# shortNames allow for shorter strings to match your resource on the CLI.
shortNames:
- ct
Apply this with kubectl apply -f crontab-crd.yaml, and just like that, your Kubernetes cluster now understands a new API endpoint: /apis/stable.example.com/v1/namespaces/default/crontabs/. You can now create a custom resource that uses this definition.
apiVersion: stable.example.com/v1
kind: CronTab
metadata:
name: my-cron-tab
spec:
cronSpec: "* * * * *"
image: my-awesome-cron-image
replicas: 2
And you can interact with it using kubectl: kubectl get ct will actually work. Magic. Or, you know, engineering.
Why the Schema is Your Best Friend and Worst Enemy
The OpenAPI schema isn’t just a suggestion; it’s a contract and a validation layer. This is where most people shoot themselves in the foot. A weak schema leads to garbage data in your cluster, which leads to your controller code panicking because it expected an integer but got the string "five".
Be ruthless with your schema. Mark fields as required if they’re non-negotiable. Use pattern for strings to enforce regex. Use minimum/maximum values for integers. The Kubernetes API server will reject any custom resource that doesn’t conform to this schema before it even gets stored in etcd. This saves you from writing a ton of boilerplate validation code later.
The flip side is that changing a schema after you have data in production is a nightmare. This is where versioning comes in.
Versioning and Storage: The Unsexy But Critical Part
Notice the versions array in the CRD? You can have multiple versions (e.g., v1alpha1, v1beta1, v1) and mark which ones are served by the API and which one is the storage version. This is Kubernetes’s way of letting you evolve your API without breaking existing clients.
You might start with a v1alpha1 version that has a field called cronSpec. Later, you realize the name schedule is more standard. You can create a v1beta1 version with the new schedule field and mark the old cronSpec as deprecated. The key is that you set one version as the storage: true version. All objects are converted to and stored in this single version. The API server can serve them in any other served: true version, handling the conversion on the fly.
This is powerful but complex. The conversion is either done by a webhook you write (for complex changes) or, more commonly, by a None strategy which relies on the fact that differing versions have the same structured data. Plan your fields carefully from the start to avoid the pain of writing a conversion webhook.
The Operator Pattern: CRDs Need a Brain
Here’s the dirty little secret they don’t tell you: a CRD by itself is useless. It’s just a fancy data store. It defines the what, but it doesn’t do the how. Creating a DatabaseCluster custom resource doesn’t magically provision a database. This is the crucial part everyone misses.
The CRD is just the API. The operator is the controller that watches for these custom resources and takes action. It’s the brain that says, “Oh, I see someone created a DatabaseCluster named prod-db. I’d better go spin up three Postgres pods, a config map, and a service.” It’s a control loop that constantly compares the desired state (what you wrote in your CronTab YAML) with the actual state of the cluster, and then makes changes to align the two.
The operator pattern is the marriage of a CRD (the declarative API) and a controller (the imperative logic). The CRD without its operator is a car without an engine. It looks right, but it’s not going anywhere.