Introduction

Kubernetes offers a variety of resource types to manage different workloads and use cases. One such resource type is DaemonSet, which ensures that a copy of a specific pod runs on all or selected nodes in a cluster. This article explores DaemonSets, their benefits, use cases, and provides practical examples.

What is a DaemonSet?

A DaemonSet is a Kubernetes resource that ensures a specified pod runs on all (or some) nodes in a cluster. Whenever a new node is added to the cluster, the DaemonSet automatically deploys a pod on that node. Conversely, when a node is removed from the cluster, the DaemonSet ensures that the corresponding pod is also removed.

Key Features of DaemonSets

1. Uniform Deployment: Ensures that all or selected nodes run the specified pods.

2. Automated Management: Automatically handles pod deployment on node addition and removal.

3. Configuration Consistency: Ensures consistent configuration and behavior across all nodes.

Pros of Using DaemonSets

• Simplified Management: Automates the deployment and management of pods across nodes.

• Consistency: Ensures that all nodes have the necessary components for monitoring, logging, or other node-specific tasks.

• Scalability: Automatically handles pods on newly added nodes without manual intervention.

• Resource Efficiency: Efficiently manages node-level workloads without the need for separate deployment configurations.

Use Cases for DaemonSets

DaemonSets are ideal for scenarios where certain tasks or services need to be present on all or specific nodes, such as:

• Logging: Deploying log collectors like Fluentd or Logstash to gather logs from all nodes.

• Monitoring: Running monitoring agents like Prometheus Node Exporter or Datadog Agent on each node.

• Networking: Ensuring network components like Calico or Weave Net are present on all nodes.

• Security: Deploying security agents or tools like Falco or antivirus solutions on each node.

Practical Examples of DaemonSets

Example 1: Deploying Fluentd for Logging

Fluentd is a popular log collector used to gather and forward logs. Here's how to deploy Fluentd using a DaemonSet.

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd
  labels:
    app: fluentd
spec:
  selector:
    matchLabels:
      name: fluentd
  template:
    metadata:
      labels:
        name: fluentd
    spec:
      containers:
      - name: fluentd
        image: fluent/fluentd:v1.11
        resources:
          limits:
            memory: 200Mi
            cpu: 0.5
          requests:
            memory: 100Mi
            cpu: 0.2
        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
      terminationGracePeriodSeconds: 30
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers

Deploy the DaemonSet using:

kubectl apply -f fluentd-daemonset.yaml

Example 2: Deploying Prometheus Node Exporter for Monitoring

Prometheus Node Exporter collects and exposes hardware and OS metrics. Here's how to deploy it using a DaemonSet.

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter
  labels:
    app: node-exporter
spec:
  selector:
    matchLabels:
      app: node-exporter
  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      containers:
      - name: node-exporter
        image: prom/node-exporter:v1.1.2
        ports:
        - containerPort: 9100
          name: metrics
        resources:
          requests:
            memory: 30Mi
            cpu: 100m
          limits:
            memory: 50Mi
            cpu: 200m
      volumes:
      - name: proc
        hostPath:
          path: /proc
          type: Directory
      - name: sys
        hostPath:
          path: /sys
          type: Directory
      - name: root
        hostPath:
          path: /
          type: Directory
        mountPropagation: HostToContainer
      containers:
      - name: node-exporter
        image: prom/node-exporter:v1.1.2
        volumeMounts:
        - name: proc
          mountPath: /host/proc
          readOnly: true
        - name: sys
          mountPath: /host/sys
          readOnly: true
        - name: root
          mountPath: /rootfs
          readOnly: true
        securityContext:
          runAsUser: 65534
          procMount: Default
        args:
        - '--path.procfs=/host/proc'
        - '--path.sysfs=/host/sys'
        - '--collector.filesystem.ignored-mount-points="^/(sys|proc|dev|host|etc)($$|/)"'

Deploy the DaemonSet using:

kubectl apply -f node-exporter-daemonset.yaml

Example 3: Deploying Calico for Networking

Calico is a popular network plugin for Kubernetes. Here's how to deploy it using a DaemonSet.

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: calico-node
  namespace: kube-system
  labels:
    k8s-app: calico-node
spec:
  selector:
    matchLabels:
      k8s-app: calico-node
  template:
    metadata:
      labels:
        k8s-app: calico-node
    spec:
      hostNetwork: true
      containers:
      - name: calico-node
        image: calico/node:v3.17.2
        env:
        - name: DATASTORE_TYPE
          value: "kubernetes"
        - name: CALICO_NETWORKING_BACKEND
          value: "bird"
        - name: CLUSTER_TYPE
          value: "k8s,bgp"
        - name: IP
          value: "autodetect"
        - name: CALICO_IPV4POOL_CIDR
          value: "192.168.0.0/16"
        - name: FELIX_IPINIPMTU
          value: "1440"
        - name: CALICO_IPV4POOL_IPIP
          value: "Always"
        securityContext:
          privileged: true
        volumeMounts:
        - mountPath: /var/run/calico
          name: var-run-calico
        - mountPath: /lib/modules
          name: lib-modules
          readOnly: true
      volumes:
      - name: var-run-calico
        hostPath:
          path: /var/run/calico
      - name: lib-modules
        hostPath:
          path: /lib/modules

Deploy the DaemonSet using:

kubectl apply -f calico-daemonset.yaml

Conclusion

DaemonSets in Kubernetes provide an efficient way to ensure that specific pods run on all or selected nodes, offering automated management, consistency, and scalability. They are particularly useful for deploying logging, monitoring, networking, and security agents across all nodes. The practical examples provided demonstrate how to deploy Fluentd, Prometheus Node Exporter, and Calico using DaemonSets, highlighting their effectiveness in managing node-level workloads in Kubernetes. Understanding and leveraging DaemonSets can significantly enhance the management and operation of your Kubernetes cluster.