Right, so you’ve got your DaemonSet humming along, deploying its little pod on every node. That’s great, until you realize you don’t actually want it on every node. Maybe you’ve got a special node reserved for massive batch jobs and your logging sidecar would just get in the way. Or perhaps you only want your fancy GPU monitoring agent on the nodes that actually have, you know, GPUs.

This is where we stop the blunt-force “deploy everywhere” approach and start getting surgical. The two primary tools for this are nodeSelector and nodeAffinity. One is a simple, no-nonsense hammer; the other is a finely-tuned scalpel. You need to know how to wield both.

The Simple Hammer: nodeSelector

This is the OG method, and it’s glorously straightforward. It’s a simple key-value pair matcher. You label your node with something like hardware: gpu, and then you tell your DaemonSet, “Only run on nodes wearing this particular nametag.”

First, you gotta tag your node. Let’s say you have a node with a couple of NVIDIA cards you want to monitor.

kubectl label nodes <your-node-name> hardware=gpu

Now, you wire that label into your DaemonSet’s pod spec. It’s just a field in the template. No frills, no fuss.

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nvidia-gpu-monitor
spec:
  selector:
    matchLabels:
      name: nvidia-gpu-monitor
  template:
    metadata:
      labels:
        name: nvidia-gpu-monitor
    spec:
      nodeSelector:
        hardware: gpu
      containers:
      - name: monitoring-agent
        image: nvidia/dcgm:latest

See that? nodeSelector: {hardware: gpu}. The scheduler looks at that, looks at the node’s labels, and if it’s not an exact match, it moves on. It’s binary. It works. But it’s also a bit… limited. What if you need more complex logic, like “run on GPU nodes OR on nodes labeled for high-performance workloads”? Enter the scalpel.

The Finely-Tuned Scalpel: nodeAffinity

nodeSelector is fine for simple “equals” matches, but the real world is messy. nodeAffinity is here for that mess. It gives you expressive operators like In, NotIn, Exists, and even allows you to set soft preferences (preferredDuringSchedulingIgnoredDuringExecution) alongside hard requirements (requiredDuringSchedulingIgnoredDuringExecution). That mouthful is Kubernetes for “give it a shot, but if you can’t, it’s fine” vs. “this is law, do not proceed without it.”

Let’s say you have a more complex scenario: you want your pod to run on nodes that are either GPU-equipped or are labeled as dedicated=monitoring. But you absolutely, positively want to avoid any node labeled environment=dev, because the monitoring traffic would overwhelm them.

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: advanced-monitor
spec:
  selector:
    matchLabels:
      name: advanced-monitor
  template:
    metadata:
      labels:
        name: advanced-monitor
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: hardware
                operator: In
                values:
                - gpu
              - key: dedicated
                operator: In
                values:
                - monitoring
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            preference:
              matchExpressions:
              - key: disktype
                operator: In
                values:
                - ssd
      containers:
      - name: monitor
        image: my-app:monitor-v2

Let’s break this down. The requiredDuringScheduling... section is the law. The nodeSelectorTerms are OR’d together, and the matchExpressions inside them are AND’d. So this reads: “Schedule this pod onto a node that EITHER has a label hardware=gpu OR has a label dedicated=monitoring.”

The preferredDuringScheduling... section is a soft preference. Here, we’re saying, “If all else is equal, and you have a choice between nodes, please for the love of all that is holy put me on a node with disktype=ssd. It’s not a deal-breaker, but it’ll make my life much easier.” The weight field is a number between 1-100 that lets you prioritize between multiple preferences.

The Gotchas: Taints and Tolerations

Here’s the part everyone forgets until 2 AM when their DaemonSet is mysteriously not scheduled on three specific nodes: nodeSelector and nodeAffinity attract pods to nodes. Taints repel pods from nodes.

A node can have a taint that says, “No pods allowed unless they explicitly tolerate my particular brand of misery.” Your DaemonSet’s pod spec needs a tolerations section to handle this. The control plane nodes are the most common example; they almost always have a node-role.kubernetes.io/control-plane:NoSchedule taint.

If you want a DaemonSet to run on the master nodes (e.g., for a networking pod), you must teach it to tolerate that taint.

tolerations:
- key: node-role.kubernetes.io/control-plane
  operator: Exists
  effect: NoSchedule

You slap that into your pod spec, and suddenly the gates to the master node open. It’s a critical piece of the puzzle. You can have the most perfectly matching nodeAffinity in the world, but if the node has a taint your pod can’t tolerate, it’s a hard stop. Always check kubectl describe node <node-name> for the Taints section when your DaemonSet is being stubborn.

The combination of attraction (nodeAffinity) and repulsion (Tolerations) is what gives you ultimate control. Use them together to place your DaemonSet pods exactly where they need to be, and nowhere they don’t.