Right, so the default scheduler is pretty good at its job, but let’s be honest: it’s a generalist. It’s designed to make pretty okay decisions for most people. But your cluster isn’t “most people.” You have weird, specific needs. Maybe you need to schedule pods based on custom hardware flags, tie them to a specific internal corporate policy, or—and I’ve seen this—make sure your batch processing jobs never run on a node named after someone’s pet cat, “Mr. Whiskers.” (Don’t ask.)

When the default kube-scheduler feels like a rented tuxedo—close enough but never a perfect fit—it’s time to talk about taking matters into your own hands. You have two primary paths: go nuclear with a full Custom Scheduler, or opt for the more surgical approach using Scheduler Plugins. One is like building a new car from scratch; the other is like swapping out the engine of your existing one.

The Nuclear Option: A Full Custom Scheduler

A custom scheduler is just that: a program you write that watches the API for unscheduled pods (pod.spec.schedulerName is set to your scheduler’s name) and then binds them to a node. It’s a complete replacement. You’re saying, “I’ll handle this from here, thanks.”

Under the hood, it’s a control loop:

  1. Watch: Use a client-go informer to watch for Pods in all namespaces where spec.schedulerName is my-custom-scheduler and spec.nodeName is empty.
  2. Filter: Run your logic to filter out ineligible nodes (this is your Filter phase).
  3. Score: Rank the remaining nodes (this is your Score phase).
  4. Bind: Choose the best node and patch the Pod’s spec.nodeName field.

Here’s a terrifyingly simplified version in Go:

package main

import (
    "context"
    "fmt"
    "log"
    "time"

    v1 "k8s.io/api/core/v1"
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    "k8s.io/client-go/kubernetes"
    "k8s.io/client-go/tools/clientcmd"
)

func main() {
    config, _ := clientcmd.BuildConfigFromFlags("", "/path/to/kubeconfig")
    clientset, _ := kubernetes.NewForConfig(config)

    podInformer := // ... setup an informer to watch for your pods ...

    for pod := range podInformer {
        if pod.Spec.SchedulerName != "my-custom-scheduler" {
            continue
        }

        nodes, _ := clientset.CoreV1().Nodes().List(context.TODO(), metav1.ListOptions{})
        var feasibleNodes []v1.Node

        // Filter Phase: Your custom logic here
        for _, node := range nodes.Items {
            if node.Name != "mr-whiskers" { // Our critical business logic
                feasibleNodes = append(feasibleNodes, node)
            }
        }

        // Score Phase: Pick one, any one. This is a bad scheduler.
        chosenNode := feasibleNodes[0]

        // Bind Phase
        binding := &v1.Binding{
            ObjectMeta: metav1.ObjectMeta{Name: pod.Name, UID: pod.UID},
            Target:     v1.ObjectReference{Kind: "Node", Name: chosenNode.Name},
        }
        err := clientset.CoreV1().Pods(pod.Namespace).Bind(context.TODO(), binding, metav1.CreateOptions{})
        if err != nil {
            log.Fatal(err)
        }
        fmt.Printf("Placed pod %s on node %s\n", pod.Name, chosenNode.Name)
    }
}

The monumental downside here is obvious: you are now responsible for every single scheduling primitive. Need node affinity? You code it. Taints and tolerations? You code it. Resource requests? You code it. You’ve thrown out decades of collective engineering effort. It’s a powerful but brutal approach, best reserved for highly specialized, non-standard workloads where the default semantics simply don’t apply.

The Surgical Option: Scheduler Plugins

This is the grown-up way to do it. Instead of reinventing the wheel, you extend it. The Kubernetes scheduler framework allows you to write plugins that hook into the existing scheduling lifecycle—PreFilter, Filter, PreScore, Score, Bind, etc. Your plugin gets called alongside the default ones.

This is how you tell the default scheduler: “Hey, you know that Filter phase you run? Run my custom filter logic as well. And when you’re scoring, here’s an extra scoring rule from me.”

You implement these plugins by writing a Go program that implements the plugin interfaces and compiles into a separate binary. The real magic is in your scheduler configuration (often a ConfigMap), where you define the scheduling profile and tell the scheduler which plugins to enable and in what order.

apiVersion: kubescheduler.config.k8s.io/v1beta3
kind: KubeSchedulerConfiguration
profiles:
  - schedulerName: default-scheduler-with-my-twist
    plugins:
      filter:
        enabled:
          - name: "NodeMrWhiskersFilter" # Your custom plugin!
      score:
        enabled:
          - name: "NodeMrWhiskersFilter"
            weight: 42 # Because it's very important
    pluginConfig:
      - name: "NodeMrWhiskersFilter"
        args:
          forbiddenNodeName: "mr-whiskers"

The massive advantage is that you get all the default functionality for free. Your plugin only needs to worry about your specific business logic. You’re injecting wisdom, not rebuilding the brain.

Which One Should You Actually Use?

Unless you’re a platform team at a hyperscaler, the answer is almost always Scheduler Plugins.

  • Custom Schedulers are a maintenance nightmare. You must keep pace with Kubernetes API changes and your scheduler becomes a critical single point of failure. If it crashes, nothing gets scheduled.
  • Scheduler Plugins are sustainable. They leverage the battle-tested default codepath. The default scheduler’s health checks, its leader election, its performance optimizations—all that still works. Your plugin is just along for the ride, adding its two cents.

The pitfall with plugins is the build and deployment process. You’re no longer just deploying manifests; you’re building, packaging, and deploying a binary that must exist on the control plane nodes and be referenced by the scheduler’s startup command. It’s an operational lift, but it’s a one-time cost for a vastly superior architecture.

So, before you embark on writing a full custom scheduler, ask yourself: “Do I truly need a new car, or do I just need a better GPS?” Nine times out of ten, it’s the GPS. Use plugins.