Helm upgrade fails with “field is immutable” after changing Deployment selector labels
Helm upgrade fails with “field is immutable” after changing Deployment selector labels
A Helm immutable Deployment selector change just blocked your rollout: helm upgrade exits with Deployment.apps "my-deploy" is invalid: spec.selector: Invalid value: ...: field is immutable, and CI stops before the new pods exist. The cost is not just a failed job; you now have a release in a failed state, a chart that no longer matches the live object, and pressure to “fix it manually” before the next deploy.
The cause is narrow: in apps/v1, a Deployment’s spec.selector is immutable [link needed]. Helm normally upgrades by patching the existing Deployment [link needed], so changing selector labels in the chart produces a patch Kubernetes rejects.
Confirm the selector is what changed
Do this before deleting anything.
1. Back up the release values and live object:
```bash
helm get values my-release -n my-ns -o yaml > values.backup.yaml
kubectl get deployment my-deploy -n my-ns -o yaml > deploy.backup.yaml
```
2. Compare live selector labels with the rendered chart:
```bash
kubectl get deployment my-deploy -n my-ns \
-o jsonpath='{.spec.selector.matchLabels}{"\n"}'
helm template my-release ./chart -n my-ns -f values.yaml \
| kubectl diff -f -
```
3. Confirm the diff touches spec.selector.matchLabels, not only pod-template labels or metadata labels. If only metadata.labels changed, recreate is usually unnecessary.
Fix a Helm immutable Deployment selector change with delete-and-recreate
The reliable fix is to let Helm create a new Deployment instead of patching the old selector. Choose one path:
Path A — accept brief downtime. Delete the Deployment, then upgrade:
kubectl delete deployment my-deploy -n my-ns --cascade=foreground
helm upgrade my-release ./chart -n my-ns -f values.yaml
kubectl rollout status deployment/my-deploy -n my-ns
Path B — keep old pods running during replacement. Orphan the dependents, upgrade, then clean up:
kubectl delete deployment my-deploy -n my-ns --cascade=orphan
helm upgrade my-release ./chart -n my-ns -f values.yaml
kubectl rollout status deployment/my-deploy -n my-ns
kubectl get rs,pods -n my-ns -l app.kubernetes.io/instance=my-release
--cascade=orphan leaves ReplicaSets and Pods behind [link needed]. After the new rollout is healthy, delete old orphaned ReplicaSets/Pods deliberately; otherwise they keep consuming capacity. If the upgrade fails after deletion, inspect helm status and either fix values and rerun helm upgrade or restore with your backup manifests.
Use helm upgrade --force only when replacement is acceptable
helm upgrade --force uses a replacement strategy for resources that need it [link needed]. It can get past the same immutability error, but treat it as a bigger hammer: it may recreate resources beyond the Deployment. Use it when you have already verified the chart, have capacity, and can tolerate the replacement.
helm upgrade my-release ./chart -n my-ns -f values.yaml --force
Do not rely on kubectl edit to change the selector on a live Deployment; the API server still requires recreation, and manual edits leave Helm drift.
When NOT to do this
Do not delete-and-recreate just because labels look different. Skip it when the selector is unchanged and only pod-template labels moved. Also pause if Services, NetworkPolicies, PodDisruptionBudgets, or monitoring selectors depend on the old pod labels: a Helm immutable Deployment selector change can silently detach traffic or alerts until those selectors are updated. For single-replica critical workloads without spare capacity, schedule the orphan path rather than deleting with foreground cascade during peak traffic.
Pre-flight checklist
- [ ] Selector diff confirmed in rendered output.
- [ ] Values and live Deployment backed up.
- [ ] Service/NetworkPolicy/PDB selectors checked against new labels.
- [ ] Capacity or maintenance window agreed.
- [ ] Recreate path chosen: foreground or orphan.
- [ ] Rollout verified, orphaned ReplicaSets cleaned up.