Kubernetes on AWS
Amazon EKS, one question at a time.
Answer one. If you miss it, a short walkthrough shows you why. If you get it, try the next.
61 concepts 58 walkthroughs 8 troubleshooting playbooks 14 field guides 15-part fleet series
Today's question · Control Plane and Data Plane
A new question each day
On EKS, which of these is your job rather than AWS's?
Pick an answer. No sign-up, nothing saved.
AWS runs the control plane. The data plane is yours: node capacity, node updates and what runs on them.
See it happen · Control Plane and Data Plane walkthrough, 11 stepsOr go straight to
Learn EKS
Start learningEvery core concept as a walkthrough with a quiz, in learning order.
Start with
Fix something
All playbooksRun it at scale
Fleet series14 production field guides, and a 15-part series on running EKS by the hundred.
Browse everything
- Playbook Troubleshooting Pod stuck in Pending 0/3 nodes are available: 3 Insufficient cpu. Open playbook
- Playbook Troubleshooting CrashLoopBackOff and OOMKilled Back-off restarting failed container Open playbook
- Playbook Troubleshooting ImagePullBackOff Failed to pull image Open playbook
- Playbook Troubleshooting Node NotReady or won't join NotReady Open playbook
- Playbook Troubleshooting Service not reachable connection refused Open playbook
- Playbook Troubleshooting DNS failures no such host Open playbook
- Playbook Troubleshooting ALB 5xx and unhealthy targets 502 Bad Gateway Open playbook
- Playbook Troubleshooting Out of Pod IPs failed to assign an IP address to container Open playbook
- You run kubectl apply. What happens before a container starts, and who runs it? What breaks first: 5 machines for 2 nodes Control Plane and Data Plane · Essentials Find out
- web runs on your laptop but crashes on node-1. How do images fix that? What breaks first: Works on my machine Container Images and Amazon ECR · Essentials Find out
- An app and a log shipper share one Pod. Why are no logs shipped? What breaks first: Containers can't share files Pods · Essentials Find out
- You want 3 copies of web, always. What keeps them running through a rollout? What breaks first: A Pod disappears Deployments and ReplicaSets · Essentials Find out
- Someone types app=Web in a Service. Why do all its endpoints vanish? What breaks first: One letter off, zero endpoints Labels, Selectors and Namespaces · Essentials Find out
- Pods keep getting new IPs. How does web keep finding shop? What breaks first: Pod IPs keep changing Services and kube-proxy · Essentials Find out
- Why does web try four wrong names before api.stripe.com? What breaks first: Four wasted lookups CoreDNS · Solid Find out
- shop-a needs 90 seconds to start, but liveness kills it at 30. Why? What breaks first: Killed before it boots Liveness, Readiness and Startup Probes · Essentials Find out
- db needs 3 Postgres Pods, each with its own data. Why not a Deployment? What breaks first: One shared claim can't spread StatefulSets · Solid Find out
- The log agent must run on every node. Why does node-3 get none? What breaks first: A Deployment misses a node DaemonSets · Solid Find out
- The nightly sales report should run once and exit. How do you schedule it? What breaks first: A Deployment never stops Jobs and CronJobs · Solid Find out
- The sales-etl driver can't start its executors. What does Spark on EKS need? What breaks first: Driver can't create executors Spark and EMR on EKS · Deep Find out
- A t3.medium stops at 17 Pods. Where do Pod IPs come from? What breaks first: node-1 hits max pods VPC CNI · Essentials Find out
- Why does EKS need a Service, a load balancer and then an Ingress? What breaks first: Pod replaced, IP breaks Ingress · Essentials Find out
- A debug Pod in another namespace reaches db. How do you lock it down? What breaks first: Every Pod can reach db Network Policies · Solid Find out
- Two teams share one ALB and shop gets 404. Who should own the listener? What breaks first: Shared ALB stops updating Gateway API · Deep Find out
- checkout calls inventory in another VPC with the same CIDR. How does it connect? What breaks first: Two VPCs, one CIDR Amazon VPC Lattice · Deep Find out
- Only payments should reach payments-db. Why can web get in too? What breaks first: Node SG lets everyone in Security Groups for Pods · Deep Find out
- You limit shop-prod's API to the office. Why does node-1 go NotReady? What breaks first: node-1 locked out, NotReady Private Clusters and Endpoint Access · Solid Find out
- Players in LA feel lag from Oregon. How do you move game-server closer? What breaks first: Players far from Oregon Local Zones and Outposts · Deep Find out
- node-1 stops at 29 Pods. Why doesn't raising max pods fix it? What breaks first: web-30 stuck in ContainerCreating Prefix Delegation and Custom Networking · Deep Find out
- web calls orders in plain HTTP. How does a mesh encrypt and check callers? What breaks first: Plain calls, no identity Service Mesh · Deep Find out
- Your nodes register but stay NotReady. What does the node role need? What breaks first: Nodes stuck NotReady Managed Node Groups · Essentials Find out
- Factory servers must join shop-prod. How do on-site machines become EKS nodes? What breaks first: Plant network isn't listed EKS Hybrid Nodes · Deep Find out
- orders-legacy is a .NET Framework app. How does it run beside Linux Pods? What breaks first: Linux nodes can't run it Windows Containers · Deep Find out
- web's Pods land on costly GPU nodes. How do you save them for train? What breaks first: Ordinary Pods take GPU nodes Taints, Tolerations and Affinity · Solid Find out
- llm asks for a GPU on gpu-1 and stays Pending. Why? What breaks first: Pending: no GPU advertised GPU Nodes and Model Serving · Deep Find out
- One GPU fails on day 5. How does atlas keep training? What breaks first: One GPU stops all 128 SageMaker HyperPod · Deep Find out
- web's traffic jumps every evening. How does it get more Pods, then fewer? What breaks first: Fixed at 3 replicas Horizontal Pod Autoscaler · Solid Find out
- 300 orders wait in SQS but CPU reads 30%. What should scale orders-worker? What breaks first: 300 waiting, CPU at 30% KEDA · Solid Find out
- 4 new web Pods sit Pending and no node appears. Who adds one? What breaks first: Pending, and no new node Cluster Autoscaler and Karpenter · Solid Find out
- batch fills a GPU node's spare CPU. How do you keep GPUs for inference? What breaks first: One NodePool for everything Karpenter NodePools and Disruption · Deep Find out
- web runs 6 Pods on full-price x86 nodes. How do you pay less? What breaks first: All On-Demand, all x86 Spot and Graviton · Solid Find out
- web moves off its node group. What does AWS run, and what breaks? What breaks first: No DaemonSet, no logs Fargate and EKS Auto Mode · Solid Find out
- Draining node-1 takes 2 of web's 3 Pods at once. What keeps web up? What breaks first: Drain takes 2 of 3 PodDisruptionBudgets · Solid Find out
- node-1 is nearly idle, yet report stays Pending. What is the scheduler counting? What breaks first: Pending on an idle node Requests, Limits, QoS and Evictions · Deep Find out
- Priya's kubectl apply gets Unauthorized, then Forbidden. What does each one mean? What breaks first: Signed in, but Unauthorized RBAC · Essentials Find out
- Sam needs kubectl, and the creator role is deleted. Who can let him in? What breaks first: Nobody can let Sam in Access Entries · Solid Find out
- orders must read S3. How does one Pod get its own AWS role? What breaks first: Pods borrow the node's role EKS Pod Identity and IRSA · Solid Find out
- A debug Pod runs privileged with its node's disk mounted. What should stop it? What breaks first: No labels, nothing checked Pod Security Admission · Deep Find out
- checkout:1.8 ships with an old log4j inside. What catches it, before and after deploy? What breaks first: Scan misses old log4j Runtime and Image Security · Solid Find out
- shop-dev runs a Pod from a Docker Hub image. What stops it? What breaks first: Nothing checks the image Policy as Code · Deep Find out
- An auditor asks if shop-prod passes CIS. How do you prove it? What breaks first: Fixed on one node only CIS Benchmarks and Compliance · Deep Find out
- web needs a database host and password. Why not bake them into the image? What breaks first: Config baked into the image ConfigMaps and Secrets · Essentials Find out
- checkout's database password sits in 3 repos. How do Pods get it without Git? What breaks first: A password in Git Secrets from AWS · Deep Find out
- reports saves PDFs to /data. How do they survive a Pod replacement? What breaks first: Files die with the Pod Persistent Volumes and the EBS CSI Driver · Solid Find out
- media's Pods run in 3 zones. How can they all share uploads? What breaks first: One EBS volume, one zone EFS and Shared Storage · Solid Find out
- web's YAML is copied per environment and prod lags. How does a chart help? What breaks first: Copied YAML drifts Helm · Solid Find out
- dev runs web:1.5, prod still runs web:1.4. How do you stop copies drifting? What breaks first: Copied YAML drifts Kustomize · Solid Find out
- CI deploys web with cluster-admin keys. How do you deploy without handing them out? What breaks first: CI holds admin keys Argo CD and GitOps · Solid Find out
- Sam edits web by hand. Does Flux put Git's version back? What breaks first: A hand edit drifts Flux · Solid Find out
- checkout:2.0 fails 5% of payments. How do you expose only a few shoppers first? What breaks first: Rollout hits every shopper Progressive Delivery · Deep Find out
- orders' bucket and database arrive days later by ticket. Can they live in Git? What breaks first: Bucket and DB by ticket ACK, kro and EKS Capabilities · Deep Find out
- kube-proxy is three versions behind. Who keeps shop-prod's core add-ons current? What breaks first: Self-managed copies drift EKS Add-ons · Solid Find out
- checkout breaks at 2 a.m. and the logs are gone. What changed? What breaks first: kubectl sees only now Observability · Solid Find out
- Finance asks what each team costs on shop-prod. Where does that number come from? What breaks first: Bill shows services, not teams Cost Visibility and Allocation · Solid Find out
- checkout must survive Friday's sale. How do you prove it survives losing a node? What breaks first: Resilient only on paper Failure Testing with AWS FIS · Deep Find out
- shop-prod runs Kubernetes 1.35. How long can it stay there, and what then? What breaks first: Extended support costs 6x Cluster Upgrades · Deep Find out
- Field guide Delivery Reaching a Managed Argo CD on Amazon EKS Click through the access path to a managed Argo CD: the endpoint, the VPC endpoint, IAM Identity Center, the capability… Explore it
- Field guide Networking Your Compute Scaled Out, Now DNS Is Timing Out Change ndots, NodeLocal DNSCache and CoreDNS sizing, and watch one lookup turn into eight DNS queries. Replay the incident
- Field guide Security Decoupling Network Ingress Click through an Edge account and a Compute account: Route 53, WAF, ACM, the ALB, the STS hop, the network join and the… Explore it
- Field guide Resilience DiskPressure Evicted Your Database to Reclaim a Few Gigabytes of Logs Toggle ephemeral-storage requests, PriorityClass and limits on three Pods, split the disks, and watch who the kubelet… Replay the incident
- Field guide Compute & scaling Why Your P99 Latency Spikes on EKS While CPU Sits at 40% Move the CPU limit, thread count and node size and watch one 100 ms CFS period: the quota runs out in milliseconds and… Replay the incident
- Field guide Delivery GitOps Drift Control That Survives a 2 a.m. Incident Give a secret, a feature flag, HPA replicas and a hotfix image each a home (Git, External Secrets, a flag store,… Replay the incident
- Field guide Networking The Custom Networking Trap Toggle custom networking, SNAT, rp_filter per interface, return routes and TGW appliance mode, and watch one SYN-ACK… Replay the incident
- Field guide Security Goodbye IRSA, Hello EKS Pod Identity Answer five questions about where your Pods run and how your clusters change, and get Pod Identity, IRSA (IAM Roles for… Make the call
- Field guide Resilience Surviving an AZ Blackout Pick a design, a failure and a fencing move, and see whether payments-0 recovers, hangs about six minutes or risks its… Replay the incident
- Field guide Compute & scaling The Spot Storm Set NodePool breadth, a pause-pod buffer, on-demand fallback and the interruption queue, then replay the same Spot storm… Replay the incident
- Field guide Networking Beyond the Defaults Pick an instance type, turn on prefix delegation, custom networking and calculator max-pods, and watch the node's ENI… Calculate it
- Series Part 1 Managing Hundreds of EKS Clusters The opener to an 18-part series on running Amazon EKS at fleet scale. Read part 1
- Series Part 2 Many Small EKS Clusters or a Few Large Ones? The Fleet Decision Part 2 of the fleet-scale EKS series, and the first hard architecture decision — the one every later article assumes… Read part 2
- Series Part 3 Provisioning an EKS Fleet Part 3 of the fleet-scale EKS series. Read part 3
- Series Part 4 GitOps at Fleet Scale Part 4 of the fleet-scale EKS series — the operational heart of the golden path. Read part 4
- Series Part 5 Golden Paths for an EKS Fleet Part 5 of the fleet-scale EKS series. Read part 5
- Series Part 6 Policy-as-Code Across an EKS Fleet Part 6 of the fleet-scale EKS series — the guardrail layer of the golden path. Read part 6
- Series Part 7 Designing a Single EKS Cluster Right Part 7 of the fleet-scale EKS series. Read part 7
- Series Part 8 EKS Networking at Fleet Scale Part 8 of the fleet-scale EKS series. Read part 8
- Series Part 9 Connecting an EKS Fleet Part 9 of the fleet-scale EKS series. Read part 9
- Series Part 10 EKS Identity at Fleet Scale Part 10 of the fleet-scale EKS series. Read part 10
- Series Part 11 EKS Secrets and Supply Chain at Fleet Scale Part 11 of the fleet-scale EKS series — closing the security triad. Read part 11
- Series Part 12 Autoscaling an EKS Fleet Part 12 of the fleet-scale EKS series. Read part 12
- Series Part 13 Observing an EKS Fleet Part 13 of the fleet-scale EKS series. Read part 13
- Series Part 14 Upgrading an EKS Fleet Part 14 of the fleet-scale EKS series. Read part 14
- Series Part 15 What an EKS Fleet Actually Costs Part 15 of the fleet-scale EKS series. Read part 15
- Field guide Worked example Cartwright on EKS Click through one enterprise app on EKS: three repos, CI gates, ECR, an Argo CD hub and an account and cluster per… Explore it
- Field guide Delivery GitOps at Fleet Scale Answer five questions about your fleet and get an Argo CD hub, the managed EKS Capability, Flux in every cluster or a… Make the call
- Field guide Delivery Argo CD Core Architecture Click any part of Argo CD's control plane to see what it does and how it scales, then kill it to see what breaks. Explore it
Nothing matches. Try another word, or clear the filter.