Service Discovery

Why This Comes Up in Interviews

Any microservices design triggers: "How does Service A find Service B?" This appears in:

• "How do your services communicate?" → Need to locate instances first
• "What happens when you scale up/down?" → New instances need to be discoverable
• "How do you handle instance failures?" → Dead instances must be removed from routing
• "How does your load balancer know the backends?" → Service registry feeds it

Understanding the trade-offs (CP vs AP, client-side vs server-side, DNS vs registry) separates junior from senior answers.

The Problem: Nothing Stays Still

In microservices, a single request might hit 5-10 services. Each service has 50+ instances spread across availability zones. Instances are created and destroyed by auto-scalers every few minutes. Deploys roll out new containers while killing old ones.

The anti-pattern that never dies: Hardcoding IP addresses in config files. One infrastructure change, one VM migration, one container restart — and half your services send requests to an IP that no longer exists.

DNS-Based Discovery — The Simplest Approach

How: Register services as DNS records. user-service.internal.company.com resolves to healthy instance IPs.

The killer problem: TTL caching.

Still used: Kubernetes (CoreDNS), Consul (DNS interface), AWS Route 53 service discovery. The trick is layering health checks and faster propagation on top.

Client-Side vs Server-Side Discovery

Industry trend: Moving toward server-side discovery via service meshes (Istio, Linkerd). The sidecar proxy handles discovery — application code doesn't know it's happening.

Service Registries Compared

ZooKeeper — The Original Coordinator

• Built at Yahoo (late 2000s) for distributed coordination, not discovery specifically
• Ephemeral znodes: Service creates a node; if heartbeat stops → node auto-deleted → watchers notified
• CP system (ZAB consensus) — during partition, minority side becomes unavailable
• Operationally complex: sessions, watches, recipes, JVM GC tuning

etcd — Kubernetes' Brain

• Distributed KV store using Raft consensus
• Leases: Write key with TTL → auto-expires if not renewed → watchers notified
• Simpler API than ZooKeeper (HTTP/gRPC)
• ALL Kubernetes state lives in etcd (pods, services, deployments, secrets)

Consul — Built for Service Discovery

• HashiCorp. The ONLY registry in this list built specifically for service discovery from day one
• Combines: service registry + health checking + KV store + multi-datacenter
• Raft (servers) + Gossip/SWIM (agents) — detects failure in <2 seconds across 10,000 nodes
• Speaks DNS natively: user-svc.service.consul resolves to healthy instances
• Rich health checks: HTTP, TCP, gRPC, TTL, script

Eureka — Netflix's AP Registry

• Deliberately AP (available during partitions, eventually consistent)
• Clients cache registry locally, refresh every 30 seconds
• Self-preservation: When too many instances fail heartbeats simultaneously (probably network issue), Eureka stops evicting and preserves last known good state
• If ALL Eureka servers die, clients work from local cache

Registry Comparison Table

Kubernetes Service Discovery

In Kubernetes, service discovery is built-in and invisible:

How it works:

1. Create a Service YAML: name: user-svc, selector matches pods with label app=user-svc
2. Kubernetes assigns a virtual ClusterIP (e.g., 10.96.0.42)
3. CoreDNS creates: user-svc.default.svc.cluster.local → 10.96.0.42
4. kube-proxy programs iptables: 10.96.0.42 → round-robin to actual pod IPs
5. Pods scale up/down → Endpoints auto-updated → routing adjusts

Why this is powerful: Application code just calls http://user-svc:8080/api — no discovery library, no registry client, no configuration. DNS + virtual IP handles everything.

Service Mesh — Discovery in the Infrastructure

With Istio/Envoy, every pod gets a sidecar proxy:

Key insight: Move service discovery from application code to infrastructure. Application doesn't know it's happening.

Health Checking Patterns

Kubernetes distinction:

• Liveness: Failed → kubelet restarts the pod
• Readiness: Failed → pod removed from Service endpoints (no traffic)
• Startup: Gives slow-starting apps time before liveness kicks in

Back-of-Envelope: Discovery Scale

Interview Framework

When asked "How do services find each other in your design?":

Step 1 — Environment: "On Kubernetes, I'd use the built-in service discovery — CoreDNS resolves service names to ClusterIPs, kube-proxy handles routing. No additional infrastructure needed."

Step 2 — Health checking: "Services expose /health endpoints. Kubernetes readiness probes remove unhealthy pods from the Service's endpoint list within seconds."

Step 3 — Service mesh (if needed): "For advanced requirements (mTLS, canary routing, observability), I'd add a service mesh (Istio). Envoy sidecars handle discovery, retries, and traffic splitting transparently."

Step 4 — Cross-cluster/region: "For multi-region, I'd use [Consul with WAN federation / global load balancer with DNS-based routing]. Each region has its own registry for local availability."

Step 5 — AP vs CP: "I'd prefer an AP discovery system — stale data is better than no data. Client-side caching ensures services can still communicate even if the registry is briefly unavailable."

Quick Recall