Circuit Breakers & Bulkheads

Why This Comes Up in Interviews

Any microservices design has dependencies. Interviewers test:

• "What happens if your payment service is slow?" → Without circuit breakers, everything dies
• "How do you prevent cascading failures?" → Circuit breaker + bulkhead
• "What's your retry strategy?" → Exponential backoff with jitter
• "How do you handle partial failures gracefully?" → Fallbacks + degraded mode

If you design a system where one slow service can take down everything else, you've failed the interview.

The Cascading Failure Problem

The chain reaction:

1. Service B becomes slow (DB overloaded, GC pause, network issue)
2. Service A calls B → threads wait for B's response (30s timeout)
3. A's thread pool fills up (all threads waiting for B)
4. A can't serve ANY requests (not just ones needing B)
5. Services that depend on A also start failing
6. Entire system goes down — from ONE slow dependency

Back-of-envelope:

Result: A goes from 2,000 req/sec to ZERO in under a second. Not because A is broken — because B is slow.

Circuit Breaker Pattern — Three States

State Transitions

CLOSED ──[failures > threshold]──→ OPEN
                                      │
                                [timeout expires]
                                      │
                                      ▼
                                  HALF-OPEN
                                   /      \
                    [probes succeed]        [probes fail]
                         │                      │
                         ▼                      ▼
                      CLOSED                   OPEN

Configuration Parameters

Bulkhead Pattern — Isolate the Blast Radius

Concept: Like a ship's bulkheads — watertight compartments that prevent one breach from sinking the entire vessel.

Without bulkhead: All dependencies share one thread pool. One slow dependency consumes all threads → entire service dies.

With bulkhead: Each dependency gets its own isolated resource pool. One slow dependency only exhausts ITS pool → other dependencies unaffected.

Implementation Patterns

Thread Pool Sizing

Formula: threads = peak_requests_per_sec × p99_latency_seconds × safety_factor

Retry with Exponential Backoff + Jitter

Why Fixed Retries Kill Systems

Scenario: Service B goes down for 1 second. 1,000 requests fail simultaneously. All 1,000 retry at exactly the same time → thundering herd → B gets slammed with 2x normal load the instant it recovers → may go down again.

The Formula

delay = min(base × 2^attempt + random_jitter, max_delay)

Why jitter is critical: Without jitter, all clients retry at exactly t=1s, t=2s, t=4s — still synchronized. With random jitter, retries spread out over the window. The thundering herd becomes a trickle.

Jitter strategies:

Timeout Patterns

The infinite timeout trap: Default HTTP clients often have NO timeout. A single stuck request holds a thread forever. Always set explicit timeouts.

Rule of thumb: Timeout should be slightly above the p99 latency of the downstream service. If p99 = 2s, set timeout = 3s.

Fallback Strategies

Real Implementations Compared

Library vs Infrastructure

Combining Patterns — The Full Defense

A production service typically layers all patterns together:

Request
  → Timeout (5s max)
    → Circuit Breaker (fail fast if dependency is down)
      → Bulkhead (don't exhaust all threads)
        → Retry (exponential backoff + jitter, max 3 attempts)
          → Actual call to dependency
            → Fallback (if all retries fail or circuit is open)

Order matters: Circuit breaker OUTSIDE retries (don't retry if circuit is open). Timeout OUTSIDE everything (cap total wait time).

Interview Framework

When asked "How do you handle failures in your distributed system?":

Step 1 — Identify the risk: "Any synchronous dependency can fail or become slow. Without protection, a slow [payment/recommendation/search] service will exhaust our thread pool and take down our entire API."

Step 2 — Circuit breaker: "I'd wrap calls to [dependency] in a circuit breaker. After [5] failures in [10] seconds, the circuit opens and requests fail immediately — protecting our thread pool. After [30]s, it enters half-open and probes for recovery."

Step 3 — Bulkhead: "Each dependency gets its own thread pool / semaphore limit. If [dependency] is slow, only its allocated threads are consumed — other dependencies continue working normally."

Step 4 — Retry + backoff: "On transient failures, I'd retry with exponential backoff and jitter: delay = base × 2^attempt + random. Max 3 retries. This prevents thundering herds on recovery."

Step 5 — Fallback: "When the circuit is open, I'd serve [cached data / default response / degraded experience] rather than a hard error. The user gets a degraded but functional experience."

Quick Recall