Kubernetes (K8s)

Master Kubernetes from cluster internals to production operations. Covers architecture, scheduling algorithms, networking model, storage, RBAC, Helm, debugging playbooks, and every question that comes up at senior-level DevOps and SRE interviews.

How to Use This Guide

This guide is structured as a progressive deep-dive, from foundational motivation through production-grade operations:

• Section 1 — Why K8s exists: the problems it solves (self-healing, scaling, zero-downtime, config management).
• Sections 2-3 — Architecture & core objects: control plane internals, worker nodes, the full kubectl apply flow, Pods, Deployments, Services, Ingress, HPA.
• Sections 4-6 — Advanced concepts: StatefulSets, DaemonSets, networking model, storage, RBAC.
• Sections 7-8 — Production: Helm, operations playbook, debugging techniques.
• Interview Q&A — FAANG-level design and concept questions with concise answers.

Start with Section 1 if you want the mental model for why orchestration exists. If you already understand the motivation, jump to Section 2 for architecture or directly to the Interview Q&A for rapid prep.

1. Why Kubernetes Exists

Kubernetes was born from Borg — Google's internal system that manages billions of containers across millions of machines. In 2014, Google open-sourced the concepts as Kubernetes. Today it runs everything from Spotify to Airbnb to every major bank.

2. Architecture — How The Cluster Works

%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '13px', 'fontFamily': 'Inter, -apple-system, sans-serif'}, 'flowchart': {'nodeSpacing': 30, 'rankSpacing': 50, 'padding': 12, 'curve': 'basis'}, 'sequence': {'actorMargin': 60, 'messageMargin': 40}, 'class': {'padding': 12}}}%%
flowchart TB
    USER["kubectl / CI/CD / Dashboard"] -->|"HTTPS"| API

    subgraph CP["Control Plane (Master Nodes)"]
        API["kube-apiserver<br/><small>the ONLY entry point</small>"]
        ETCD[("etcd cluster<br/><small>all cluster state<br/>strongly consistent</small>")]
        SCHED["kube-scheduler<br/><small>pod → node matching</small>"]
        CM["controller-manager<br/><small>reconciliation loops</small>"]
        CCM["cloud-controller-manager<br/><small>LBs, disks, node lifecycle</small>"]

        API <--> ETCD
        SCHED --> API
        CM --> API
        CCM --> API
    end

    subgraph WN1["Worker Node 1"]
        KL1["kubelet<br/><small>pod lifecycle agent</small>"]
        KP1["kube-proxy<br/><small>iptables/IPVS rules</small>"]
        CR1["containerd<br/><small>container runtime</small>"]
        P1["Pod A"] & P2["Pod B"]
    end

    subgraph WN2["Worker Node 2"]
        KL2["kubelet"]
        KP2["kube-proxy"]
        CR2["containerd"]
        P3["Pod C"] & P4["Pod D"] & P5["Pod E"]
    end

    API --> KL1 & KL2
    KL1 --> CR1
    KL2 --> CR2

    style CP fill:#EDE9FE,stroke:#8B5CF6,color:#5B21B6
    style WN1 fill:#D1FAE5,stroke:#10B981,color:#065F46
    style WN2 fill:#DBEAFE,stroke:#3B82F6,color:#1E40AF

Control Plane Deep Dive

Worker Node Deep Dive

The Full Request Flow: kubectl apply -f deployment.yaml

3. Core Objects — Pods, Deployments, Services

K8s Resource Map

Nodes & Pods — The Physical View

A Node is a physical or virtual machine in the cluster. Each node runs multiple Pods, and each pod contains one or more Containers. This is the fundamental deployment unit hierarchy:

%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '13px', 'fontFamily': 'Inter, -apple-system, sans-serif'}, 'flowchart': {'nodeSpacing': 30, 'rankSpacing': 50, 'padding': 12, 'curve': 'basis'}, 'sequence': {'actorMargin': 60, 'messageMargin': 40}, 'class': {'padding': 12}}}%%
flowchart TB
    subgraph NODE1["Node 1 (Worker)"]
        direction TB
        subgraph P1["Pod: order-api"]
            C1["container 1<br/><small>order-service :8080</small>"]
            C2["container 2<br/><small>envoy-sidecar :15001</small>"]
        end
        subgraph P2["Pod: payment-api"]
            C3["container 3<br/><small>payment-service :8080</small>"]
        end
        subgraph P3["Pod: cart-api"]
            C4["container 4<br/><small>cart-service :8080</small>"]
            C5["container 5<br/><small>redis-cache :6379</small>"]
        end
    end

    subgraph NODE2["Node 2 (Worker)"]
        direction TB
        subgraph P4["Pod: user-api"]
            C6["container 6<br/><small>user-service :8080</small>"]
        end
        subgraph P5["Pod: notification-api"]
            C7["container 7<br/><small>notification-svc :8080</small>"]
            C8["container 8<br/><small>kafka-consumer :9092</small>"]
        end
        subgraph P6["Pod: search-api"]
            C9["container 9<br/><small>search-service :8080</small>"]
        end
    end

    style NODE1 fill:#FFCDD2,stroke:#E53935,color:#B71C1C
    style NODE2 fill:#E8EAF6,stroke:#5C6BC0,color:#1A237E
    style P1 fill:#EF9A9A,stroke:#E53935,color:#B71C1C
    style P2 fill:#EF9A9A,stroke:#E53935,color:#B71C1C
    style P3 fill:#EF9A9A,stroke:#E53935,color:#B71C1C
    style P4 fill:#9FA8DA,stroke:#5C6BC0,color:#1A237E
    style P5 fill:#9FA8DA,stroke:#5C6BC0,color:#1A237E
    style P6 fill:#9FA8DA,stroke:#5C6BC0,color:#1A237E

Key relationships:

Why Pods wrap Containers (not deploy containers directly):

• Containers in the same pod share localhost — no network hop between sidecar and main app
• Shared volumes — log shipper reads files written by the app container
• Atomic scheduling — "these two containers MUST be on the same node"
• Lifecycle coupling — if the main container dies, the whole pod restarts

Pods — The Atomic Unit

%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '13px', 'fontFamily': 'Inter, -apple-system, sans-serif'}, 'flowchart': {'nodeSpacing': 30, 'rankSpacing': 50, 'padding': 12, 'curve': 'basis'}, 'sequence': {'actorMargin': 60, 'messageMargin': 40}, 'class': {'padding': 12}}}%%
flowchart TB
    subgraph POD["Pod (IP: 10.244.1.8)"]
        direction LR
        MAIN["Main Container<br/><small>your-api :8080</small>"]
        SIDE["Sidecar<br/><small>envoy-proxy :15001</small>"]
        INIT["Init Container<br/><small>db-migrator (ran first, then exited)</small>"]
    end

    VOL["Shared Volume<br/>/data"]
    NET["Shared Network Namespace<br/><small>localhost:8080 ↔ localhost:15001</small>"]

    POD --- VOL
    POD --- NET

    style POD fill:#DBEAFE,stroke:#3B82F6,color:#1E40AF
    style VOL fill:#D1FAE5,stroke:#10B981,color:#065F46
    style NET fill:#FEF3C7,stroke:#F59E0B,color:#92400E

Multi-container patterns (these come up in FAANG interviews):

Probes — The Self-Healing Mechanism:

Deployments — The Workload Manager

A Deployment manages ReplicaSets, which manage Pods. You declare desired state, K8s makes it happen.

%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '13px', 'fontFamily': 'Inter, -apple-system, sans-serif'}, 'flowchart': {'nodeSpacing': 30, 'rankSpacing': 50, 'padding': 12, 'curve': 'basis'}, 'sequence': {'actorMargin': 60, 'messageMargin': 40}, 'class': {'padding': 12}}}%%
flowchart TB
    DEP["Deployment<br/><small>api-server</small>"]
    DEP --> RS1["ReplicaSet (current)<br/><small>api-server-7d8f6<br/>replicas: 3</small>"]
    DEP -.-> RS2["ReplicaSet (previous)<br/><small>api-server-5c9a2<br/>replicas: 0 (scaled down)</small>"]
    RS1 --> P1["Pod"] & P2["Pod"] & P3["Pod"]

    style DEP fill:#8B5CF6,stroke:#6D28D9,color:#fff
    style RS1 fill:#D1FAE5,stroke:#10B981,color:#065F46
    style RS2 fill:#FEE2E2,stroke:#EF4444,color:#991B1B

Rolling Update — How Zero-Downtime Works

%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '13px', 'fontFamily': 'Inter, -apple-system, sans-serif'}, 'flowchart': {'nodeSpacing': 30, 'rankSpacing': 50, 'padding': 12, 'curve': 'basis'}, 'sequence': {'actorMargin': 60, 'messageMargin': 40}, 'class': {'padding': 12}}}%%
flowchart LR
    subgraph BEFORE["Before (v1 running)"]
        direction TB
        V1A["v1 pod A ✓"]
        V1B["v1 pod B ✓"]
        V1C["v1 pod C ✓"]
    end

    subgraph DURING["Mid-rollout"]
        direction TB
        V1X["v1 pod A ✓"]
        V2Y["v2 pod B ✓<br/><small>readiness passed</small>"]
        V1Z["v1 pod C<br/><small>draining...</small>"]
    end

    subgraph AFTER["After (v2 complete)"]
        direction TB
        V2A["v2 pod A ✓"]
        V2B["v2 pod B ✓"]
        V2C["v2 pod C ✓"]
    end

    BEFORE --> DURING --> AFTER

    style BEFORE fill:#DBEAFE,stroke:#3B82F6,color:#1E40AF
    style DURING fill:#FEF3C7,stroke:#F59E0B,color:#92400E
    style AFTER fill:#D1FAE5,stroke:#10B981,color:#065F46

The rolling update cycle for each pod:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-server
spec:
  replicas: 3
  revisionHistoryLimit: 5
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  selector:
    matchLabels: { app: api-server }
  template:
    metadata:
      labels: { app: api-server, version: v2.1.0 }
    spec:
      terminationGracePeriodSeconds: 30
      containers:
        - name: api
          image: registry.example.com/api:v2.1.0@sha256:abc123...
          ports: [{ containerPort: 8080 }]
          resources:
            requests: { memory: "256Mi", cpu: "250m" }
            limits: { memory: "512Mi", cpu: "1000m" }
          readinessProbe:
            httpGet: { path: /ready, port: 8080 }
            initialDelaySeconds: 5
            periodSeconds: 5
            failureThreshold: 3
          livenessProbe:
            httpGet: { path: /healthz, port: 8080 }
            initialDelaySeconds: 15
            periodSeconds: 10
          lifecycle:
            preStop:
              exec:
                command: ["/bin/sh", "-c", "sleep 5"]

Services — Stable Networking Abstraction

%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '13px', 'fontFamily': 'Inter, -apple-system, sans-serif'}, 'flowchart': {'nodeSpacing': 30, 'rankSpacing': 50, 'padding': 12, 'curve': 'basis'}, 'sequence': {'actorMargin': 60, 'messageMargin': 40}, 'class': {'padding': 12}}}%%
flowchart TB
    subgraph EXTERNAL["External Traffic"]
        CLIENT["Browser / Mobile"]
    end

    subgraph CLUSTER["Cluster"]
        LB["LoadBalancer Service<br/><small>provisions AWS ALB/NLB</small><br/><small>External IP: 52.x.x.x</small>"]
        NP["NodePort<br/><small>:30080 on every node</small>"]
        CIP["ClusterIP Service<br/><small>api-service.default.svc.cluster.local</small><br/><small>10.96.0.42:80</small>"]

        CIP --> P1["Pod 10.244.1.8"]
        CIP --> P2["Pod 10.244.2.3"]
        CIP --> P3["Pod 10.244.3.7"]
    end

    subgraph INTERNAL["Internal Service-to-Service"]
        APP["order-service pod"]
        CIP2["ClusterIP: payment-service<br/><small>10.96.0.88:80</small>"]
        PAY1["payment pod 1"]
        PAY2["payment pod 2"]
    end

    CLIENT --> LB --> NP --> CIP
    APP -->|"http://payment-service:80"| CIP2
    CIP2 --> PAY1 & PAY2

    style LB fill:#EDE9FE,stroke:#8B5CF6,color:#5B21B6
    style CIP fill:#D1FAE5,stroke:#10B981,color:#065F46
    style CIP2 fill:#D1FAE5,stroke:#10B981,color:#065F46

Service types — know when to use each:

Ingress — Production HTTP Routing

%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '13px', 'fontFamily': 'Inter, -apple-system, sans-serif'}, 'flowchart': {'nodeSpacing': 30, 'rankSpacing': 50, 'padding': 12, 'curve': 'basis'}, 'sequence': {'actorMargin': 60, 'messageMargin': 40}, 'class': {'padding': 12}}}%%
flowchart LR
    INTERNET["Internet"] --> IC["Ingress Controller<br/><small>(nginx / traefik / ALB)</small>"]
    IC -->|"api.myapp.com/v1/*"| API["api-service :8080"]
    IC -->|"api.myapp.com/auth/*"| AUTH["auth-service :8080"]
    IC -->|"docs.myapp.com/*"| DOCS["docs-service :3000"]
    IC -->|"ws.myapp.com"| WS["websocket-service :9090"]

    style IC fill:#EDE9FE,stroke:#8B5CF6,color:#5B21B6
    style API fill:#D1FAE5,stroke:#10B981,color:#065F46
    style AUTH fill:#FEF3C7,stroke:#F59E0B,color:#92400E
    style DOCS fill:#DBEAFE,stroke:#3B82F6,color:#1E40AF
    style WS fill:#FEE2E2,stroke:#EF4444,color:#991B1B

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-ingress
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/rate-limit: "100"
spec:
  ingressClassName: nginx
  tls:
    - hosts: [api.myapp.com]
      secretName: api-tls-cert
  rules:
    - host: api.myapp.com
      http:
        paths:
          - path: /v1
            pathType: Prefix
            backend:
              service: { name: api-service, port: { number: 80 } }
          - path: /auth
            pathType: Prefix
            backend:
              service: { name: auth-service, port: { number: 80 } }

HPA — Automatic Scaling

%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '13px', 'fontFamily': 'Inter, -apple-system, sans-serif'}, 'flowchart': {'nodeSpacing': 30, 'rankSpacing': 50, 'padding': 12, 'curve': 'basis'}, 'sequence': {'actorMargin': 60, 'messageMargin': 40}, 'class': {'padding': 12}}}%%
flowchart LR
    MS["Metrics Server<br/><small>scrapes kubelet every 15s</small>"] --> HPA["HPA Controller"]
    HPA -->|"desired = ceil(current × actual/target)"| DEP["Deployment<br/><small>replicas: 3 → 8</small>"]
    DEP --> RS["ReplicaSet"]
    RS --> P1["Pod"] & P2["Pod"] & P3["Pod"]
    RS -.->|"scaling up"| P4["Pod 4"] & P5["Pod 5"]

    style HPA fill:#F59E0B,stroke:#92400E,color:#fff
    style P4 fill:#D1FAE5,stroke:#10B981,color:#065F46
    style P5 fill:#D1FAE5,stroke:#10B981,color:#065F46

How HPA calculates replicas:

desiredReplicas = ceil( currentReplicas × (currentMetricValue / targetMetricValue) )

Example: 3 replicas at 90% CPU, target is 60% → ceil(3 × 90/60) = ceil(4.5) = 5 replicas

Scaling behaviors:

• Scale up: Fast (30s stabilization window). React quickly to load spikes.
• Scale down: Slow (5 min stabilization). Avoid thrashing on bursty traffic.

4. Advanced Workloads — StatefulSets, DaemonSets, Jobs

StatefulSets — When Identity Matters

StatefulSet guarantees:

%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '13px', 'fontFamily': 'Inter, -apple-system, sans-serif'}, 'flowchart': {'nodeSpacing': 30, 'rankSpacing': 50, 'padding': 12, 'curve': 'basis'}, 'sequence': {'actorMargin': 60, 'messageMargin': 40}, 'class': {'padding': 12}}}%%
flowchart LR
    SS["StatefulSet: postgres"] --> P0["postgres-0<br/><small>pvc: data-postgres-0 (100Gi)</small>"]
    SS --> P1["postgres-1<br/><small>pvc: data-postgres-1 (100Gi)</small>"]
    SS --> P2["postgres-2<br/><small>pvc: data-postgres-2 (100Gi)</small>"]
    HS["Headless Service"] --> DNS0["postgres-0.postgres-headless<br/>.default.svc.cluster.local"]
    HS --> DNS1["postgres-1.postgres-headless"]
    HS --> DNS2["postgres-2.postgres-headless"]

    style SS fill:#EDE9FE,stroke:#8B5CF6,color:#5B21B6
    style P0 fill:#D1FAE5,stroke:#10B981,color:#065F46
    style P1 fill:#D1FAE5,stroke:#10B981,color:#065F46
    style P2 fill:#D1FAE5,stroke:#10B981,color:#065F46

• Created in order: 0 → 1 → 2 (each must be Ready before next starts)
• Terminated in reverse: 2 → 1 → 0
• Each pod's PVC persists even if the pod is deleted and rescheduled to another node
• Headless service gives each pod its own DNS name

DaemonSets — One Pod Per Node

Need something running on EVERY node? Monitoring agent, log collector, network plugin, storage daemon.

%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '13px', 'fontFamily': 'Inter, -apple-system, sans-serif'}, 'flowchart': {'nodeSpacing': 30, 'rankSpacing': 50, 'padding': 12, 'curve': 'basis'}, 'sequence': {'actorMargin': 60, 'messageMargin': 40}, 'class': {'padding': 12}}}%%
flowchart TB
    DS["DaemonSet: node-exporter"]
    DS --> N1["Node 1<br/>node-exporter pod"]
    DS --> N2["Node 2<br/>node-exporter pod"]
    DS --> N3["Node 3<br/>node-exporter pod"]
    DS -.->|"new node added"| N4["Node 4<br/>auto-created pod"]

    style DS fill:#F59E0B,stroke:#92400E,color:#fff
    style N4 fill:#D1FAE5,stroke:#10B981,color:#065F46

Common DaemonSets in production: Prometheus Node Exporter, Fluentd/Filebeat, Datadog Agent, Calico/Cilium (CNI), CSI node driver

Jobs & CronJobs

5. Networking — Pod-to-Pod, Services, Network Policies

The K8s Networking Model (Three Rules)

1. Every pod gets its own IP (no port conflicts between pods)
2. All pods can reach all other pods without NAT (flat network — any pod on any node talks to any other pod on any other node)
3. Agents on a node can reach all pods on that node

This flat network is implemented by CNI plugins: Calico (BGP), Cilium (eBPF), Flannel (VXLAN), AWS VPC CNI.

How Pod-to-Pod Works (Cross-Node)

%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '13px', 'fontFamily': 'Inter, -apple-system, sans-serif'}, 'flowchart': {'nodeSpacing': 30, 'rankSpacing': 50, 'padding': 12, 'curve': 'basis'}, 'sequence': {'actorMargin': 60, 'messageMargin': 40}, 'class': {'padding': 12}}}%%
flowchart LR
    subgraph NODE1["Node 1 (10.0.1.10)"]
        PA["Pod A<br/>10.244.1.8"]
        CB1["cbr0 bridge"]
    end

    subgraph NODE2["Node 2 (10.0.1.11)"]
        PB["Pod B<br/>10.244.2.3"]
        CB2["cbr0 bridge"]
    end

    PA --> CB1 -->|"VXLAN tunnel / BGP route"| CB2 --> PB

    style NODE1 fill:#DBEAFE,stroke:#3B82F6,color:#1E40AF
    style NODE2 fill:#D1FAE5,stroke:#10B981,color:#065F46

Network Policies — Pod-Level Firewall

%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '13px', 'fontFamily': 'Inter, -apple-system, sans-serif'}, 'flowchart': {'nodeSpacing': 30, 'rankSpacing': 50, 'padding': 12, 'curve': 'basis'}, 'sequence': {'actorMargin': 60, 'messageMargin': 40}, 'class': {'padding': 12}}}%%
graph LR
    API["api pods<br/><small>label: app=api</small>"] -->|"ALLOWED on :5432"| DB["postgres pods<br/><small>label: app=postgres</small>"]
    FE["frontend pods"] -->|"ALLOWED on :8080"| API
    FE -.->|"DENIED"| DB
    OTHER["any other pod"] -.->|"DENIED"| DB

    style DB fill:#EDE9FE,stroke:#8B5CF6,color:#5B21B6
    style OTHER fill:#FEE2E2,stroke:#EF4444,color:#991B1B

6. Storage & Persistence

%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '13px', 'fontFamily': 'Inter, -apple-system, sans-serif'}, 'flowchart': {'nodeSpacing': 30, 'rankSpacing': 50, 'padding': 12, 'curve': 'basis'}, 'sequence': {'actorMargin': 60, 'messageMargin': 40}, 'class': {'padding': 12}}}%%
flowchart LR
    POD["Pod"] -->|"volumeMount"| PVC["PVC<br/><small>'give me 100Gi fast SSD'</small>"]
    PVC -->|"bound"| PV["PV<br/><small>actual provisioned disk</small>"]
    SC["StorageClass<br/><small>'fast-ssd' → gp3, 5000 IOPS</small>"] -->|"dynamic provisioning"| PV
    PV -->|"backed by"| DISK["AWS EBS / GCP PD / Azure Disk"]

    style PVC fill:#DBEAFE,stroke:#3B82F6,color:#1E40AF
    style PV fill:#D1FAE5,stroke:#10B981,color:#065F46
    style SC fill:#EDE9FE,stroke:#8B5CF6,color:#5B21B6

Access modes (determines how many nodes can mount):

7. Security — RBAC, Secrets, Pod Security

RBAC — Who Can Do What

%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '13px', 'fontFamily': 'Inter, -apple-system, sans-serif'}, 'flowchart': {'nodeSpacing': 30, 'rankSpacing': 50, 'padding': 12, 'curve': 'basis'}, 'sequence': {'actorMargin': 60, 'messageMargin': 40}, 'class': {'padding': 12}}}%%
flowchart LR
    subgraph SCOPE["Namespace: 'production'"]
        ROLE["Role: 'developer'<br/><small>pods: get, list, watch<br/>deployments: get, list, create, update</small>"]
    end

    USER["User: vamsi<br/>Group: dev-team"] -->|"RoleBinding"| ROLE

    subgraph GLOBAL["Cluster-Wide"]
        CR["ClusterRole: 'readonly'<br/><small>all resources: get, list, watch</small>"]
    end

    SRE["Group: sre-team"] -->|"ClusterRoleBinding"| CR

    style ROLE fill:#D1FAE5,stroke:#10B981,color:#065F46
    style CR fill:#EDE9FE,stroke:#8B5CF6,color:#5B21B6

Best practices:

• Principle of least privilege — start with nothing, grant only what's needed
• Namespace-scoped Roles whenever possible (not ClusterRoles)
• Use Groups, not individual Users
• Audit: kubectl auth can-i --list --as=vamsi -n production

Secrets — The Reality

K8s Secrets are base64-encoded, NOT encrypted. Anyone with kubectl get secret can decode them.

Production secret management:

8. Helm — Templating & Package Management

# Install a complex application in one command
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install monitoring prometheus-community/kube-prometheus-stack \
  --namespace monitoring --create-namespace \
  --set grafana.adminPassword=secret \
  --set prometheus.retention=30d

# Upgrade with new values
helm upgrade monitoring prometheus-community/kube-prometheus-stack \
  -f values-prod.yaml --atomic --wait

# Rollback if something breaks
helm rollback monitoring 1

CI/CD pattern: helm upgrade --install --atomic --wait — idempotent (install or upgrade), auto-rollback on failure, block until healthy.

9. Production Operations Playbook

10. Debugging Production Issues

11. Interview Q&A (FAANG-Level)

Global: Route53 / Cloud DNS (latency-based routing per region)

Per Region:
├── EKS/GKE cluster (3 AZs, managed control plane)
├── Node pools: general (m5.xlarge), memory-optimized (r5.2xlarge), GPU (p3.2xlarge)
├── Ingress: AWS ALB Ingress Controller + WAF + cert-manager
├── Service mesh: Istio (mTLS, observability, traffic management)
├── Stateless services: Deployments + HPA + PDB
├── Stateful: AWS RDS (outside K8s) / ElastiCache (outside K8s)
├── Async: Kafka on dedicated StatefulSet nodes OR MSK
├── Secrets: External Secrets Operator + AWS Secrets Manager
├── Observability: Prometheus + Grafana + Loki + Jaeger
├── GitOps: ArgoCD (app-of-apps pattern)
└── Policy: OPA Gatekeeper (enforce labels, resource limits, image sources)