Proxies

Why This Comes Up in Interviews

Proxies are rarely the main topic, but they appear in EVERY system design:

• "How do clients reach your services?" → Reverse proxy / API gateway
• "How do you do TLS termination?" → Reverse proxy
• "How do services communicate in your mesh?" → Sidecar proxy (Envoy)
• "How do you rate limit / load balance?" → Proxy layer
• "How do you do canary deployments?" → Proxy-based traffic splitting

Understanding L4 vs L7 proxy capabilities is essential for making correct architectural decisions.

Forward vs Reverse — The Key Distinction

In system design interviews, you almost always mean reverse proxy. Forward proxy is a client-side concept (VPN, corporate firewall).

Reverse Proxy — What It Actually Does in Production

L4 vs L7 Proxy — The Critical Difference

When to use L4: - Backend is not HTTP (databases, Redis, custom protocols) - Maximum performance (millions of connections) - Don't need content-based routing

When to use L7: - Need to route by URL, header, or cookie - TLS termination required - Want connection pooling to backends - A/B testing, canary routing - WAF (Web Application Firewall) rules

Real Proxy Software Compared

Proxy Patterns in System Design

Pattern 1: API Gateway

The entry point for all client requests. Combines multiple proxy responsibilities:

Real examples: Kong, AWS API Gateway, Apigee, custom Nginx/Envoy config.

Pattern 2: Service Mesh (Sidecar Proxy)

Every service gets its own proxy (Envoy sidecar):

Key insight: Move cross-cutting concerns (retries, auth, observability, traffic management) from application code to infrastructure (proxy).

Pattern 3: BFF (Backend For Frontend)

Different clients (web, iOS, Android) need different API shapes:

Why: Mobile clients need smaller payloads, different pagination, offline support. One-size-fits-all API serves nobody well.

Back-of-Envelope: Proxy Capacity

Nginx performance characteristics:

Scaling proxy layers:

Connection Pooling — Why Proxies Make Backends Faster

Without proxy: - 10,000 clients each open a connection to backend - Backend manages 10,000 connections (memory, file descriptors) - Each request = TCP handshake (if not keepalive)

With proxy (connection multiplexing): - 10,000 clients connect to proxy - Proxy maintains 100 persistent connections to backend - Multiplexes 10,000 client requests over 100 backend connections - Backend is happy (low connection count)

Real impact: PostgreSQL handles ~500 connections well. With pgbouncer (proxy): 10,000+ client connections mapped to 100 backend connections.

Proxy vs Load Balancer vs API Gateway vs CDN

In practice these overlap: Nginx can be all of the first three. Cloudflare is CDN + reverse proxy + WAF. Envoy is proxy + LB + mesh. The distinction is about PRIMARY INTENT, not capability.

Interview Framework

When discussing how clients reach your services:

Step 1 — Edge layer: "Client requests hit a [CDN for static / reverse proxy for API]. This handles TLS termination, rate limiting, and request routing."

Step 2 — Routing: "The proxy routes based on URL path: /api/users/* → user service, /api/orders/* → order service. This is an L7 decision."

Step 3 — L4 vs L7: "I'd use L7 proxy (Nginx/Envoy) because we need content-based routing. For database connections, I'd use L4 (NLB/HAProxy TCP mode) since it's not HTTP."

Step 4 — Service mesh (if microservices): "For service-to-service communication, each service has an Envoy sidecar handling mTLS, retries, and circuit breaking. This keeps cross-cutting concerns out of application code."

Step 5 — Scaling: "The proxy layer scales horizontally. At [X] RPS, we'd need [N] proxy instances behind a DNS-level load balancer."

Quick Recall