ConfigMaps and Secrets are resources in Kubernetes that are used to govern configuration for applications. ConfigMaps maintain non-sensitive data whereas Secrets hold sensitive data like credentials and passwords.
In Kubernetes 1.21, a feature for Immutability of ConfigMaps and Secrets was GA. What exactly is this concept and its purpose? This article has the answers.
Concept
Kubernetes 1.21 was released back in 2021, and it provided an option to mark ConfigMaps and Secrets as immutable. This meant that once created, the data in these objects could not be altered.
Purpose
Some would question the need for this feature. Since ConfigMaps and Secrets deal with configuration, and configuration is subject to change, why would we need such a feature?
It is important to understand that the kubelet on each node watches over each ConfigMap and Secret and periodically syncs its information to the Pods via the kube-apiserver. Large scale production deployments can have tens of thousands of such ConfigMaps and Secrets and obviously there is a cost associated with continuously keeping a watch for changes.
ConfigMaps and Secrets containing data that is constant over time can be marked as immutable. The kubelet does not watch over these resources. This in turn reduces the load on the kube-apiserver, saves compute resources and effectively improves the performance of the cluster.
Security and Reliability
As the data for immutable ConfigMaps and Secrets cannot be altered, it can protect the sensitive and critical configuration from accidental (or unwanted) updates that could cause application outages.
It provides an additional layer of security against malicious actors attempting to manipulate data.
Consistency
Immutability is ideal for scenarios where you want to ensure a consistent configuration across multiple Pods and Deployments.
Definition
Immutable ConfigMaps and Secrets are defined by setting the immutable field to true.
apiVersion: v1
kind: ConfigMap
metadata:
...
data:
...
immutable: true
apiVersion: v1
kind: Secret
metadata:
...
data:
...
immutable: true
Example
Let's check an example of an Immutable ConfigMap that is mounted as a Volume in a Pod. We will also demonstrate how to update configuration driven by an Immutable ConfigMap.
⚠
WARNING: Immutable ConfigMaps and Secrets are especially used for configuration which is constant and is not expected to change over time. In a production use-case, it is advisable to not change the contents of Immutable ConfigMaps and Secrets unless absolutely necessary.
Prerequisites
A local Kubernetes cluster such as minikube, kind, K3s or MicroK8s.
The kubectl command line tool to interact with the Kubernetes cluster.
An example manifest for an Immutable ConfigMap is shown below:
apiVersion: v1
data:
season: "summer"
kind: ConfigMap
immutable: true
metadata:
name: season
Create the Immutable ConfigMap:
kubectl apply -f https://raw.githubusercontent.com/adityasamant25/courses/main/kubernetes/configmaps/examples/immutable-configmaps.yaml
Below is an example of a Deployment manifest with the Immutable ConfigMap season mounted as a volume into the Pod's only container:
apiVersion: apps/v1
kind: Deployment
metadata:
name: immutable-configmap-volume
labels:
app.kubernetes.io/name: immutable-configmap-volume
spec:
replicas: 3
selector:
matchLabels:
app.kubernetes.io/name: immutable-configmap-volume
template:
metadata:
labels:
app.kubernetes.io/name: immutable-configmap-volume
spec:
containers:
- name: alpine
image: alpine:3
command:
- /bin/sh
- -c
- while true; do echo "$(date) My preferred season is $(cat /etc/config/season)";
sleep 10; done;
ports:
- containerPort: 80
volumeMounts:
- name: config-volume
mountPath: /etc/config
volumes:
- name: config-volume
configMap:
name: season
Create the Deployment:
kubectl apply -f https://raw.githubusercontent.com/adityasamant25/courses/main/kubernetes/configmaps/examples/deployment-with-immutable-configmap-as-volume.yaml
Check the Deployment:
kubectl get deployment immutable-configmap-volume
You should see an output similar to:
NAME READY UP-TO-DATE AVAILABLE AGE
configmap-volume 3/3 3 3 19s
Check the pods for this Deployment (matching by selector):
kubectl get pods --selector=app.kubernetes.io/name=immutable-configmap-volume
You should see an output similar to:
NAME READY STATUS RESTARTS AGE
immutable-configmap-volume-5c677f6494-n7cg5 1/1 Running 0 30s
immutable-configmap-volume-5c677f6494-sdhkp 1/1 Running 0 30s
immutable-configmap-volume-5c677f6494-vsszf 1/1 Running 0 30s
The Pod's container refers to the data defined in the ConfigMap and uses it to print a report to stdout. You can check this report by viewing the logs for one of the Pods in that Deployment:
kubectl logs deployments/immutable-configmap-volume
You should see an output similar to:
Found 3 pods, using pod/immutable-configmap-volume-5c677f6494-n7cg5
Tue Mar 5 11:38:27 UTC 2024 My preferred season is summer
Tue Mar 5 11:38:37 UTC 2024 My preferred season is summer
Tue Mar 5 11:38:47 UTC 2024 My preferred season is summer
Tue Mar 5 11:38:57 UTC 2024 My preferred season is summer
Tue Mar 5 11:39:07 UTC 2024 My preferred season is summer
Edit the ConfigMap:
kubectl edit configmap season
In the editor that appears, change the value of key season from summer to winter.
Here's an example of how that manifest could look after you edit it:
apiVersion: v1
data:
season: winter
immutable: true
kind: ConfigMap
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","data":{"season":"summer"},"immutable":true,"kind":"ConfigMap","metadata":{"annotations":{},"name":"season","namespace":"default"}}
creationTimestamp: "2024-03-05T11:36:22Z"
name: season
namespace: default
resourceVersion: "3840"
uid: 10cdc5e6-47b8-4473-a9f3-ed401c2073f9
Attempt to save the changes. You should see that the save fails and a temporary manifest is opened which contains the reason for failure:
Without any further change, save the temporary manifest. You should see an output similar to:
A copy of your changes has been stored to "/var/folders/nn/bb6y1j8d69n9lhqppjlrfthm0000gn/T/kubectl-edit-139284375.yaml"
error: Edit cancelled, no valid changes were saved.
Once a ConfigMap is marked as immutable, it is not possible to revert this change nor to mutate the contents of the data or the binaryData field. In order to modify the contents, the only option is to delete and recreate the ConfigMap.
Delete and recreate the ConfigMap season by using the temporary manifest as follows:
kubectl replace --force -f /var/folders/nn/bb6y1j8d69n9lhqppjlrfthm0000gn/T/kubectl-edit-139284375.yaml
Tail (follow the latest entries in) the logs of one of the pods that belongs to this Deployment:
kubectl logs deployments/immutable-configmap-volume --follow
Notice that the output remains unchanged, even though you recreated the ConfigMap:
Tue Mar 5 12:02:07 UTC 2024 My preferred season is summer
Tue Mar 5 12:02:17 UTC 2024 My preferred season is summer
Tue Mar 5 12:02:27 UTC 2024 My preferred season is summer
Tue Mar 5 12:02:37 UTC 2024 My preferred season is summer
⚠
CAUTION: Although the ConfigMap has been recreated, the existing Pods maintain a mount point to the deleted ConfigMap. To force an update, you would need to clean up every existing reference to the old ConfigMap. In this scenario, it equates to deleting the Deployment and recreating it. Exercise caution before executing these steps in a production environment.
🗒
A simple restart of the existing Deployment using the kubectl rollout restart command causes the Pods to be recreated, but is insufficient to refresh the contents of the ConfigMap.
Delete the Deployment:
kubectl delete deployment immutable-configmap-volume
Wait for all the Pods to terminate. Verify that the Pods have terminated using the following command:
kubectl get pods --selector=app.kubernetes.io/name=immutable-configmap-volume
You should see an output similar to:
No resources found in default namespace.
Recreate the Deployment:
kubectl apply -f https://k8s.io/examples/deployments/deployment-with-immutable-configmap-as-volume.yaml
Next, check the Deployment:
kubectl get deployment immutable-configmap-volume
You should see an output similar to:
NAME READY UP-TO-DATE AVAILABLE AGE
immutable-configmap-volume 3/3 3 3 6s
Check the Pods:
kubectl get pods --selector=app.kubernetes.io/name=immutable-configmap-volume
You should see an output similar to:
NAME READY STATUS RESTARTS AGE
immutable-configmap-volume-5c677f6494-dxjwg 1/1 Running 0 31s
immutable-configmap-volume-5c677f6494-k54fc 1/1 Running 0 31s
immutable-configmap-volume-5c677f6494-w9b4c 1/1 Running 0 31s
View the logs for a Pod in this Deployment:
# Pick one Pod that belongs to the Deployment, and view its logs
kubectl logs deployment/immutable-configmap-volume
You should see an output similar to the below:
Found 3 pods, using pod/immutable-configmap-volume-5c677f6494-dxjwg
Tue Mar 5 12:40:43 UTC 2024 My preferred season is winter
Tue Mar 5 12:40:53 UTC 2024 My preferred season is winter
Tue Mar 5 12:41:03 UTC 2024 My preferred season is winter
Summary
Once a ConfigMap is marked as immutable, it is not possible to revert this change nor to mutate the contents of the data or the binaryData field. You can only delete and recreate the ConfigMap. Because existing resources maintain a mount point to the deleted ConfigMap, it is necessary to ensure that every existing reference to the old ConfigMap (not just Pods, any reference) has been cleaned up.
Cleaning up
Delete the resources created during the tutorial:
kubectl delete deployment immutable-configmap-volume
kubectl delete configmap season
