The Service Account That Lost Its Power
Config reloader sidecar stopped updating after the cluster upgrade. Stale routing rules in production.
The Scenario
A cluster upgrade to Kubernetes 1.28 completed one hour ago. Since then, 1-in-3 requests to the routing service return 404. One pod shows Running but only 1/2 containers are Ready - the config-reloader sidecar is failing its readiness probe. The sidecar uses a ServiceAccount token to read ConfigMaps from the API server. Before the upgrade, tokens were long-lived. After the upgrade, projected tokens rotate every hour. The sidecar loaded the token at startup and never refreshed it. Now it's making authenticated API calls with an expired token.
What You'll Learn
How Kubernetes 1.21+ projected ServiceAccount tokens differ from legacy tokens
Why sidecars that cache tokens at startup break silently after cluster upgrades
Reading token expiry from JWT payloads to confirm the root cause
Implementing token rotation in sidecars using the TokenRequest API or volume reload
Tools You'll Use
Real-World Context
The switch from long-lived ServiceAccount tokens to short-lived projected tokens (default in 1.21+) has broken sidecars at hundreds of companies post-upgrade. Any sidecar that reads a token once at startup is a time bomb.
Ready to debug this?
Free account required - sign up with GitHub or Google in 10 seconds
Play The Service Account That Lost Its Power