Managing Cluster Certificates

1. Viewing Certificate Expiration

Resource	Field Path	Expiration Info	Notes
Certificate	`certificates[].notAfter`	`2025-06-15T14:30:00Z`	ISO 8601 formatted timestamp
Secret	`data.tls.crt`	`2025-06-15T14:30:00Z`	PEM encoded certificate
APIService	`spec.certificateAuthority`	`2025-06-15T14:30:00Z`	CA certificate expiration

Example: Check Certificate Expiry via kubectl

kubectl get secret -n kube-system <secret-name> -o jsonpath='{.data.tls\.crt}' | openssl x509 -inform pem -noout -enddate

Example: Using OpenSSL to Parse Cert

echo "-----BEGIN CERTIFICATE-----"<<EOF
<cert-content>
-----END CERTIFICATE-----" | openssl x509 -noout -dates

2. Renewing Certificates

Component	Renewal Method	Notes
kube-apiserver	`--client-ca-file` and `--tls-cert-file`	Requires restart to pick up new certificates
etcd	`--cert-file` and `--key-file`	Must be renewed before expiration to avoid cluster downtime
kubelet	`--tls-cert-file` and `--tls-private-key-file`	Renewal requires node restart or manual reload via API

Example: Renewing kube-apiserver Certificate

openssl req -new -x509 -key /etc/kubernetes/pki/apiserver.key \
  -out /etc/kubernetes/pki/apiserver.crt \
  -days 365 \
  -subj "/CN=kube-apiserver"

Note: After renewing certificates, restart the kube-apiserver pod or node to apply changes.

Warning: If certificates expire, the cluster becomes inaccessible until renewed.

3. Generating New Certificates

Field	Description	Example
certificates.kubernetes.io/subject	Subject of the certificate	`/CN=example.com`
certificates.kubernetes.io/issuer	Issuer of the certificate	`/O=Example Inc.`
certificates.kubernetes.io/validity	Duration of validity in seconds	`31536000`

Example: Using kubectl to generate a new certificate

kubectl create secret tls example-tls --cert=example.crt --key=example.key

Note: Ensure that the certificate and key files are in PEM format.

Warning: Do not expose private keys in plain text or version control.

4. Configuring Certificate Rotation

Configuration Option	Description	Default Value	Notes
--rotate-certificates	Enables automatic certificate rotation for the kubelet	false	Requires kubelet to be restarted after enabling
--cert-dir	Directory where certificates are stored	/var/lib/kubelet/pki	Must be accessible to kubelet
--rotate-certificates	Enables automatic certificate rotation for the API server	false	Requires API server restart to take effect

Example: Enabling Certificate Rotation for Kubelet

sudo systemctl edit kubelet

[Service]
Environment=KUBELET_EXTRA_ARGS=--rotate-certificates=true --cert-dir=/var/lib/kubelet/pki

Example: API Server Certificate Rotation Configuration

apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
metadata:
  name: config
controlPlane:
  localAPIEndpoint:
    bindPort: 6443
  certificateKey: <certificate-key>
  kubeadm:
    certificatesDir: /etc/kubernetes/pki
    clusterName: <cluster-name>
    apiServer:
      certSANs:
        - 127.0.0.1
        - localhost
      extraArgs:
        rotate-certificates: "true"

Note: Certificate rotation is managed by the kubelet and kubeadm. Ensure that the certificates are backed up before enabling rotation.

Warning: Disabling certificate rotation can lead to security vulnerabilities if certificates expire without renewal.

5. Installing cert-manager

cert-manager is a Kubernetes add-on to automate the management and issuance of TLS certificates from various issuing sources like Let's Encrypt, HashiCorp Vault, and others.

Component	Description	Version
cert-manager	Core controller for managing certificates	1.14.5
acme	ACME client for Let's Encrypt and similar services	1.14.5

Example: Installing cert-manager via Helm

helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace --version v1.14.5

Example: Installing cert-manager via kubectl

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.5/cert-manager.yaml

Note: Ensure that the namespace cert-manager exists or is created before installing cert-manager.

Warning: cert-manager requires ClusterIssuer or Issuer resources to be configured for certificate issuance.

Installation Steps

Add the Jetstack Helm repository
Update Helm repositories
Install cert-manager using Helm or kubectl
Verify installation by checking pod status in the cert-manager namespace

6. Creating Certificate Resources

Field	Description	Example
apiVersion	API version of the certificate resource	`cert-manager.io/v1`
kind	Kind of the resource	`Certificate`
metadata.name	Name of the certificate resource	`example-com-tls`
spec.secretName	Name of the secret to store the certificate	`example-com-tls`
spec.issuerRef.name	Name of the issuer to use for signing	`letsencrypt-prod`
spec.issuerRef.kind	Kind of the issuer (e.g., Issuer, ClusterIssuer)	`ClusterIssuer`
spec.dnsNames	List of DNS names to include in the certificate	`["example.com", "www.example.com"]`
spec.duration	Duration of the certificate (default 90 days)	`P90d`
spec.renewalInterval	How often to renew the certificate (default 30 days)	`P30d`

Example: Creating a Certificate Resource

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: example-com-tls
spec:
  secretName: example-com-tls
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer
  dnsNames:
    - example.com
    - www.example.com
  duration: P90d
  renewalInterval: P30d

7. Configuring Issuers

Property	Description	Values	Notes
kind	Resource type	`Issuer`, `ClusterIssuer`	Defines scope of issuer
metadata.name	Issuer name	`my-issuer`	Must be unique within namespace (for `Issuer`) or cluster (for `ClusterIssuer`)
spec.issuerRef	Reference to CA issuer	`{name: "ca-issuer", kind: "Issuer"}`	Used for chaining to existing CA
spec.acme.email	ACME account email	`admin@example.com`	Required for Let's Encrypt
spec.acme.server	ACME server URL	`https://acme-v02.api.letsencrypt.org/directory`	Production endpoint; use staging for testing
spec.acme.privateKey.secretName	Secret containing ACME key	`acme-account-key`	Must be in same namespace as `Issuer`
spec.acme.solvers	ACME challenge solvers	`[{http01: {ingress: {class: "nginx"}}}]`	Defines how to solve HTTP-01 challenges

Example: ACME Issuer Configuration

apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: admin@example.com
    privateKey:
      secretName: letsencrypt-prod-key
    solvers:
    - http01:
        ingress:
          class: nginx

Example: Self-Signed Issuer

apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: selfsigned-issuer
spec:
  selfSigned: {}

Example: ClusterIssuer for Wildcard Certificates

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: cf-certs
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: admin@example.com
    privateKey:
      secretName: cf-certs-key
    solvers:
    - http01:
        ingress:
          class: nginx

8. Using ACME Protocol

Field	Description	Values
acme.server	ACME server URL	`https://acme-v02.api.letsencrypt.org/directory`
acme.email	Contact email for ACME account	`admin@example.com`
acme.http01.port	Port for HTTP-01 challenge	`80`
acme.tls.alpn.port	Port for TLS-ALPN challenge	`443`
acme.challengeType	Challenge type to use	`http-01` or `tls-alpn-01`

Example: ACME Configuration in Kubernetes

apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: admin@example.com
    privateKeySecretRef:
      name: letsencrypt-prod-key
    http01:
      ingress:
        class: nginx

Note: Ensure that the http01 or tls-alpn-01 challenge ports are accessible from the internet for certificate issuance.

Warning: The ACME protocol requires a valid email address to register with the CA. Misconfigured emails may lead to failed certificate issuance.

9. Troubleshooting Certificate Issues

Issue	Possible Cause	Solution
Certificate expired	`cert-manager` or manually managed certs	Renew using `kubectl cert-manager renew` or update manually
Certificate not trusted	Self-signed or internal CA not added to trust store	Add CA certificate to client trust store or use `caBundle` in config
Incorrect DNS names	Subject Alternative Name (SAN) mismatch	Verify and update SANs in certificate request
Permission denied	Insufficient RBAC permissions for `cert-manager`	Check and update RBAC rules for cert-manager service account
Issuer not ready	Issuer configuration error or missing resources	Check `Issuer` and `ClusterIssuer` status with `kubectl describe`

Example: Renewing a Certificate

kubectl cert-manager renew &lt;certificate-name&gt;

Note: Use kubectl describe certificate <name> to inspect detailed status and events.

Warning: Manual renewal may cause downtime if not done carefully.

Certificate Troubleshooting Workflow

Check certificate status with kubectl get certificates
Inspect events using kubectl describe certificate <name>
Verify issuer configuration and readiness
Ensure DNS names match expected values
Validate CA trust chain if self-signed

10. Updating kubeconfig After Renewal

Example: Update kubeconfig with new certificate

kubectl config set-cluster my-cluster \
  --certificate-authority=/path/to/new-ca.crt \
  --server=https://k8s-api-server:6443

kubectl config set-credentials my-user \
  --client-certificate=/path/to/new-client.crt \
  --client-key=/path/to/new-client.key

kubectl config set-context my-context \
  --cluster=my-cluster \
  --user=my-user

kubectl config use-context my-context

Configuration Field	Description	Value Example
certificate-authority	Path to CA certificate file	`/path/to/new-ca.crt`
client-certificate	Path to client certificate file	`/path/to/new-client.crt`
client-key	Path to client key file	`/path/to/new-client.key`
server	Kubernetes API server address	`https://k8s-api-server:6443`

Note: Ensure the new certificates are valid and not expired before updating kubeconfig.

Warning: Updating kubeconfig without proper certificates can lead to authentication failures.