Skip to main content
Mapagam

Working with Operators

1. Understanding Operator Pattern

Aspect Description Details
Core Concept What it is An Operator is a method of packaging, deploying, and managing Kubernetes applications. It extends the Kubernetes API to create, configure, and manage instances of complex stateful applications like databases or monitoring systems.
Operator Lifecycle Stages
  1. Installation via CRDs (Custom Resource Definitions)
  2. Creation of Custom Resources (CRs) to define desired state
  3. Operator watches for CRs and reconciles actual state with desired state
  4. Automated actions based on changes in CRs
Types of Operators Classification
  • Cluster Operator: Manages cluster-level resources (e.g., Prometheus Operator)
  • Database Operator: Manages databases (e.g., PostgreSQL Operator)
  • Application Operator: Manages application lifecycle (e.g., Kafka Operator)

Example: Operator Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-operator
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-operator
  template:
    metadata:
      labels:
        app: my-operator
    spec:
      containers:
      - name: operator
        image: my-operator-image:latest
Note: Operators leverage Kubernetes' controller pattern to automate the management of complex applications.

Operator vs Helm Chart

  • Operator: Provides full lifecycle management, custom API, and reconciliation loop
  • Helm Chart: Provides templating and packaging but lacks custom API or reconciliation

2. Installing Operators

Method Description Requirements Notes
OperatorHub Use Operator Lifecycle Manager (OLM) to install operators from OperatorHub.io OLM installed in cluster Supports automatic updates and dependency management
OLM CLI (opm) Use Operator Package Manager to build and manage operator catalogs opm binary, operator package Advanced users and CI/CD pipelines
Helm Charts Install operators via Helm charts that include OLM manifests Helm installed, operator chart Can be integrated into existing Helm workflows
Manual Deployment Deploy operator YAMLs directly using kubectl apply kubectl, operator manifests Simplest for testing or custom deployments

Example: Installing via OperatorHub

# Install OLM
kubectl apply -f https://raw.githubusercontent.com/operator-framework/operator-lifecycle-manager/master/deploy/upstream/quickstart/crds.yaml
kubectl apply -f https://raw.githubusercontent.com/operator-framework/operator-lifecycle-manager/master/deploy/upstream/quickstart/olm.yaml

# Create subscription to operator
kubectl apply -f <operator-subscription.yaml>
Note: Ensure the namespace for the operator is created before installation.
Warning: Manual deployment bypasses OLM's dependency and update management.

3. Using Operator Lifecycle Manager

Example: Install OLM via YAML

apiVersion: v1
kind: Namespace
metadata:
  name: olm-system
---
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: operatorgroup
  namespace: olm-system
spec:
  targetNamespaces:
    - olm-system
---
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: subscription
  namespace: olm-system
spec:
  channel: stable
  installPlanApproval: Automatic
  name: prometheus
  source: operatorhubio-catalog
  sourceNamespace: olm
Property Description Values
OperatorGroup Defines the namespace scope for operators olm-system
Subscription Declares an operator to be installed prometheus
channel Release channel for operator updates stable, beta
installPlanApproval Controls how install plans are approved Automatic, Manual
Note: OLM automates operator lifecycle management including installation, updates, and removal.
Warning: Ensure targetNamespaces is correctly set to avoid permission issues.

Key Concepts

  • OperatorGroup: Manages namespace scope for operators
  • Subscription: Declares desired operator and channel
  • InstallPlan: Describes the steps to install an operator

4. Creating Custom Resources

Field Description Required Type
apiVersion API version of the custom resource Yes string
kind Kind of the custom resource Yes string
metadata.name Name of the custom resource Yes string
spec Specification of the desired state for the custom resource No object

Example: Creating a Custom Resource

apiVersion: apps.example.com/v1
kind: MyApplication
metadata:
  name: my-app
spec:
  replicas: 3
  image: nginx:1.20

5. Viewing Operator Logs

Resource Command Description
Operator Pod kubectl logs <operator-pod-name> View logs from the operator's pod
Operator Pod (previous) kubectl logs <operator-pod-name> --previous View logs from a previous instance of the pod
Operator Pod (last 100 lines) kubectl logs <operator-pod-name> --tail=100 View the last 100 lines of logs
Operator Pod (since time) kubectl logs <operator-pod-name> --since=1h View logs from the last hour
Operator Pod (follow) kubectl logs <operator-pod-name> -f Stream logs in real-time

Example: Viewing Operator Logs

kubectl logs my-operator-pod -n operator-namespace

Example: Streaming Operator Logs

kubectl logs my-operator-pod -n operator-namespace -f
Note: Ensure you have the necessary RBAC permissions to view logs from pods in the target namespace.

6. Updating Custom Resources

Field Description Example
metadata.name Name of the Custom Resource instance my-app-config
spec Desired state of the resource 
{
  "replicas": 3,
  "image": "nginx:1.21"
}
status Current state of the resource (managed by Operator) 
{
  "readyReplicas": 3,
  "conditions": [
    {
      "type": "Available",
      "status": "True"
    }
  ]
}

Example: Update Custom Resource Spec

apiVersion: apps.example.com/v1
kind: MyApplication
metadata:
  name: my-app-config
spec:
  replicas: 5
  image: nginx:1.22
Note: Changes to spec trigger reconciliation in the Operator.
Warning: Do not modify status directly as it is managed by the Operator.

7. Monitoring Operator Status

Field Description Values
status.phase Current phase of the operator Running, Failed, Unknown
status.conditions List of conditions indicating operator status Array of condition objects
status.observedGeneration Generation observed by the operator Integer
status.versions Version information of the operator Object with version details

Example: Operator Status Output

apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
  name: example-operator
spec:
  version: v1.0.0
status:
  phase: Running
  observedGeneration: 1
  versions:
    - version: v1.0.0
      image: example/operator:v1.0.0
  conditions:
    - type: Progressing
      status: "True"
      reason: OperatorIsRunning
Note: The status.phase field is a high-level indicator of the operator's operational state.
Warning: Changes to status fields are managed by the operator itself and should not be manually edited.

8. Troubleshooting Operators

Common Troubleshooting Steps

  1. Check Operator Logs
    kubectl logs -n <operator-namespace> <operator-pod-name>
  2. Verify Operator Deployment Status
    kubectl get deployments -n <operator-namespace>
  3. Inspect Operator CRD
    kubectl get crd <operator-crd-name>
  4. Check Custom Resource Status
    kubectl get <custom-resource> -n <namespace> -o yaml
  5. Validate RBAC Permissions
    kubectl auth can-i get <resource> --as <service-account>
Issue Potential Cause Solution
Operator Not Starting Missing RBAC permissions or invalid CRD Check logs and ensure proper role bindings
Custom Resource Stuck in Pending Operator not watching the CR or invalid spec Validate CRD and operator configuration
Operator Crashes Repeatedly Memory or CPU limits exceeded, or runtime errors Review logs and adjust resource limits
Note: Use kubectl describe on CRs to get detailed event information.

Operator vs Helm Chart Debugging

  • Operators: Focus on custom resource status and logs
  • Helm Charts: Focus on release status and values overrides

9. Upgrading Operators

Step Description Action
Check Current Version Identify the installed version of the operator kubectl get deployment -n <namespace> <operator-name>
Review Release Notes Check for breaking changes or new features in the new version Visit operator's official documentation or GitHub releases page
Update Operator Manifest Apply updated CRD or deployment YAML files kubectl apply -f <updated-manifest.yaml>
Verify Upgrade Ensure operator is running with the new version kubectl get pods -n <namespace>
Note: Always backup your cluster before upgrading critical operators.
Warning: Some operators may require manual steps or downtime during upgrade.

10. Deleting Operators

Step Description Action
1 Identify the Operator to be deleted Review the Operator's CRD and deployment details
2 Remove Custom Resources Delete all instances of the Operator's CRDs
3 Uninstall the Operator Delete the Operator's deployment, service, and RBAC rules
4 Remove CRDs Delete the Operator's CustomResourceDefinitions
5 Verify Cleanup Ensure no remnants of the Operator exist in the cluster

Example: Delete Operator Deployment

kubectl delete deployment operator-deployment -n operator-namespace

Example: Delete CRDs

kubectl delete crd operator-crds.example.com
Note: Ensure all Custom Resources are deleted before removing CRDs to avoid orphaned resources.
Warning: Deleting CRDs will permanently remove all associated Custom Resources. Back up data before proceeding.

Was this article helpful?

Related Cheat Sheets