Configuring Pod Affinity and Anti-Affinity

1. Using Pod Affinity

Field	Description	Example
affinity.podAffinity	Defines rules for scheduling pods on nodes based on the presence of other pods	`{ "podAffinity": { "requiredDuringSchedulingIgnoredDuringExecution": [ { "labelSelector": { "matchLabels": { "app": "frontend" } }, "topologyKey": "kubernetes.io/hostname" } ] } }`

Example: Affinity Rule for Same Node

apiVersion: v1
kind: Pod
metadata:
  name: frontend-pod
spec:
  affinity:
    podAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - labelSelector:
            matchLabels:
              app: backend
          topologyKey: kubernetes.io/hostname
  containers:
    - name: frontend
      image: nginx

Note: The topologyKey determines the scope of the affinity rule. Using kubernetes.io/hostname ensures pods are scheduled on the same node.

Example: Soft Affinity with Preferred Rules

apiVersion: v1
kind: Pod
metadata:
  name: web-pod
spec:
  affinity:
    podAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          podAffinityTerm:
            labelSelector:
              matchLabels:
                app: database
            topologyKey: kubernetes.io/zone
  containers:
    - name: web
      image: nginx

Note: Preferred rules do not enforce scheduling but increase the likelihood of pods being scheduled together.

2. Using Pod Anti-Affinity

Pod anti-affinity ensures that pods are not scheduled on the same node or in the same topology domain, which helps to distribute workloads and improve fault tolerance.

Field	Description	Values	Notes
antiAffinity	Defines anti-affinity rules for pods	`requiredDuringSchedulingIgnoredDuringExecution`, `preferredDuringSchedulingIgnoredDuringExecution`	Controls how strictly anti-affinity is enforced
labelSelector	Selector to identify pods for anti-affinity	`matchLabels`, `matchExpressions`	Used to match pods based on labels
topologyKey	Node label key to define topology domain	`kubernetes.io/hostname`, `topology.kubernetes.io/zone`	Used to define where pods should not be scheduled together

Example: Pod Anti-Affinity with topologyKey

apiVersion: v1
kind: Pod
metadata:
  name: web-pod-1
spec:
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - labelSelector:
            matchLabels:
              app: web
          topologyKey: kubernetes.io/hostname
  containers:
    - name: web-container
      image: nginx

Example: Preferred Anti-Affinity

apiVersion: v1
kind: Pod
metadata:
  name: web-pod-2
spec:
  affinity:
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          podAffinityTerm:
            labelSelector:
              matchLabels:
                app: web
            topologyKey: kubernetes.io/hostname
  containers:
    - name: web-container
      image: nginx

3. Setting Topology Keys

Field	Description	Example
topologyKey	Specifies the node label key to use for topology spread	`topology.kubernetes.io/zone`
topologyKey	Used for pod anti-affinity to spread pods across different topology domains	`kubernetes.io/hostname`

Example: Pod Affinity with Topology Key

apiVersion: v1
kind: Pod
metadata:
  name: frontend-pod
spec:
  affinity:
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchLabels:
              app: frontend
          topologyKey: topology.kubernetes.io/zone

Example: Pod Anti-Affinity with Topology Key

apiVersion: v1
kind: Pod
metadata:
  name: database-pod
spec:
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchLabels:
            app: database
        topologyKey: kubernetes.io/hostname

4. Using requiredDuringScheduling

Field	Description	Values	Notes
requiredDuringScheduling	Defines hard constraints for pod placement	`true` or `false`	When set to `true`, the scheduler will not schedule a pod unless it meets the affinity/anti-affinity rules. When `false`, it will attempt to schedule the pod but may not meet the rules.

Example: Pod Affinity with requiredDuringScheduling

apiVersion: v1
kind: Pod
metadata:
  name: pod-with-affinity
spec:
  affinity:
    podAffinity:
      requiredDuringScheduling:
        - labelSelector:
            matchLabels:
              app: frontend
          topologyKey: kubernetes.io/hostname
  containers:
    - name: app-container
      image: nginx

Example: Pod Anti-Affinity with requiredDuringScheduling

apiVersion: v1
kind: Pod
metadata:
  name: pod-with-anti-affinity
spec:
  affinity:
    podAntiAffinity:
      requiredDuringScheduling:
        - labelSelector:
            matchLabels:
              app: database
          topologyKey: kubernetes.io/hostname
  containers:
    - name: db-container
      image: postgres

5. Using preferredDuringScheduling

The preferredDuringScheduling setting in Kubernetes allows you to define soft constraints for pod placement, meaning the scheduler will try to schedule pods according to these rules but will not fail if they cannot be met.

Field	Description	Values
preferredDuringSchedulingIgnoredDuringExecution	Defines a list of affinity terms with weights	Array of `PodAffinityTerm` objects

Example: Preferred Affinity Rule

affinity:
  podAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchLabels:
              app: frontend
          topologyKey: kubernetes.io/hostname

This configuration tells the scheduler to prefer placing pods with the label app: frontend on the same node, but it's not a hard requirement. If no such nodes are available, the pods will still be scheduled.

Note: The weight field influences the preference strength, with higher weights having more impact on scheduling decisions.

6. Configuring Label Selectors

Property	Description	Values	Notes
matchLabels	Simple key-value pairs to match	`{"key": "value"}`	Used to match pods with specific labels
matchExpressions	Complex label matching with operators	`[{key: "key", operator: "In\|NotIn\|Exists\|DoesNotExist", values: ["val1", "val2"]}]`	Allows for more flexible label matching logic

Example: Using matchLabels

affinity:
  podAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchLabels:
            app: frontend
        topologyKey: kubernetes.io/hostname

Example: Using matchExpressions

affinity:
  podAntiAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchExpressions:
              - key: app
                operator: In
                values: [backend, database]
          topologyKey: failure-domain.beta.kubernetes.io/zone

Note: matchLabels and matchExpressions can be used together to define more complex selection criteria.

7. Setting Namespace Selectors

Field	Description	Values	Notes
namespaceSelector	Defines the namespace selector to match pods	`matchLabels`, `matchExpressions`	Used in conjunction with podAffinity/podAntiAffinity

Example: Using namespaceSelector in Pod Affinity

apiVersion: v1
kind: Pod
metadata:
  name: pod-with-namespace-selector
spec:
  affinity:
    podAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchExpressions:
            - key: app
              operator: In
              values: [frontend]
          namespaceSelector:
            matchLabels:
              env: production
          topologyKey: kubernetes.io/hostname

8. Understanding Topology Domains

Domain	Description	Example
kubernetes.io/hostname	Node hostname	`node-1`
topology.kubernetes.io/zone	Availability zone	`us-west-1a`
topology.kubernetes.io/region	Region	`us-west-1`
kubernetes.io/arch	Architecture	`amd64`
kubernetes.io/os	Operating system	`linux`
beta.kubernetes.io/instance-type	Instance type (deprecated)	`m5.large`

Note: Topology domains are used in pod affinity and anti-affinity rules to control how pods are scheduled across nodes.

9. Combining Affinity Rules

Example: Combining Node and Pod Affinity

apiVersion: v1
kind: Pod
metadata:
  name: pod-with-combined-affinity
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: topology.kubernetes.io/zone
            operator: In
            values:
            - us-west-1a
    podAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchLabels:
            app: frontend
        topologyKey: kubernetes.io/hostname
  containers:
  - name: app-container
    image: nginx

Rule Type	Configuration	Description
nodeAffinity	`requiredDuringSchedulingIgnoredDuringExecution`	Pods must schedule on nodes matching the specified node selector terms
podAffinity	`requiredDuringSchedulingIgnoredDuringExecution`	Pods must schedule on the same topology as matching pods

Note: Combining affinity rules allows fine-grained control over pod placement, ensuring both node and pod-level constraints are satisfied.

10. Troubleshooting Pod Placement

Steps to Troubleshoot Pod Placement Issues

Check Pod Status: Use kubectl describe pod <pod-name> to see events and reasons for scheduling failures.
Review Node Taints and Tolerations: Ensure that the pod has appropriate tolerations for any taints on nodes.
Validate Resource Requests and Limits: Confirm that the pod's resource requests are compatible with available node resources.
Analyze Affinity/Anti-Affinity Rules: Inspect the pod's affinity and anti-affinity rules to ensure they are correctly configured.
Check Node Selectors and Labels: Verify that nodes have the required labels matching pod node selectors.

Issue	Possible Cause	Solution
Pending State	`0/1 nodes are available: 1 Insufficient cpu.`	Adjust pod resource requests or add nodes with more CPU.
Scheduling Failed	`0/1 nodes are available: 1 node(s) didn't match node selector.`	Ensure pod's nodeSelector matches node labels.
Taints Prevent Scheduling	`0/1 nodes are available: 1 node(s) had taints that the pod didn't tolerate.`	Add tolerations to the pod to allow scheduling on tainted nodes.
Affinity Misconfiguration	`0/1 nodes are available: 1 node(s) didn't have labels matching pod's affinity rules.`	Verify label keys and values in pod affinity rules match node labels.

Note: Use kubectl get pods --all-namespaces -o wide to view pod placement across nodes.

Example: Describe Pod to Find Scheduling Issues

kubectl describe pod my-pod -n my-namespace