JVM Tuning, Performance & Monitoring

• -Xms sets the initial heap size. The JVM starts with this much heap allocated.
• -Xmx sets the maximum heap size. The JVM cannot grow the heap beyond this.
• -Xss sets the thread stack size for each thread (default ~512KB-1MB).

You set -Xms equal to -Xmx in production to:

1. Avoid heap resizing pauses -- The JVM does not need to request more memory from the OS during runtime
2. Predictable memory footprint -- Important for container resource limits (Kubernetes)
3. Avoid fragmentation -- The OS allocates one contiguous block upfront

Pair with -XX:+AlwaysPreTouch to fault in all pages at startup, avoiding page faults during peak load.

G1GC (Garbage-First Garbage Collector) divides the heap into equally-sized regions (1-32MB). Each region is dynamically assigned as Eden, Survivor, Old, or Humongous. G1 maintains a priority queue of regions sorted by garbage density and collects the most garbage-rich regions first.

Collection lifecycle:

1. Young GC -- Evacuates live objects from Eden/Survivor regions (STW)
2. Concurrent marking -- Triggered when heap occupancy exceeds InitiatingHeapOccupancyPercent (default 45%). Marks live objects concurrently with application threads.
3. Mixed GC -- Collects both young and old regions with high garbage
4. Full GC (fallback) -- Happens only if G1 cannot keep up; single-threaded and very slow

Tuning for latency:

• -XX:MaxGCPauseMillis=100 -- Set aggressive pause target
• -XX:InitiatingHeapOccupancyPercent=35 -- Start concurrent marking earlier to avoid Full GC
• -XX:G1HeapRegionSize=16m -- Larger regions reduce humongous allocations for large objects
• -XX:G1ReservePercent=15 -- More reserve space prevents to-space exhaustion
• Give enough heap that G1 is never pressured into Full GC

ZGC is a concurrent garbage collector that achieves sub-millisecond pause times regardless of heap size. It uses colored pointers (metadata stored in pointer bits) and load barriers (inserted by JIT at every reference load) to perform relocation concurrently with the application.

Key properties:

• Pause times are O(1) with respect to heap size -- even multi-terabyte heaps have <1ms pauses
• Since Java 21, ZGC supports generational mode, which improves throughput by separating short-lived and long-lived objects
• Memory is returned to the OS when not needed (-XX:SoftMaxHeapSize)

Choose ZGC over G1 when:

• p99 latency SLAs require sub-millisecond tail latencies
• Heap sizes exceed 16GB (G1 pause times scale with heap)
• Serving latency-sensitive workloads (ad bidding, financial services, gaming)
• Running on Java 21+ where Generational ZGC closes the throughput gap with G1

Stick with G1 when:

• Heap is moderate (2-8GB) and throughput matters more than worst-case latency
• Running on older Java versions (ZGC was experimental before Java 15)
• Your workload is batch-oriented rather than latency-sensitive

Step-by-step approach:

1. Detect -- Monitor heap usage trends via JMX/Prometheus. A leak shows a sawtooth pattern where the baseline after each Full GC steadily rises.

2. Capture evidence -- Take two heap dumps separated by time:

   Bash

   jcmd <pid> GC.heap_dump /tmp/heap1.hprof
   # Wait 10-30 minutes
   jcmd <pid> GC.heap_dump /tmp/heap2.hprof
   

3. Analyze with Eclipse MAT:

4. Open the Leak Suspects report for automatic detection
5. Compare histograms between the two dumps to identify growing classes
6. Use Dominator Tree to find which objects retain the most memory

7. Trace Path to GC Roots to find the reference chain keeping leaked objects alive

8. Common leak sources:

9. Static collections (Map, List) that grow unbounded
10. Listeners/callbacks not deregistered
11. Thread-local variables in thread pools (threads are reused, ThreadLocal is never cleared)
12. Classloader leaks in app servers with hot deploy

13. Unclosed resources (Connection, InputStream, ResultSet)

14. Fix -- Remove the strong reference, add cleanup hooks, use WeakReference where appropriate, or close resources in finally/try-with-resources.

JFR is a low-overhead (<2%) profiling and event recording framework built into the JVM. It records events about GC, CPU, memory, I/O, threads, locks, and custom application events into a binary .jfr file.

Production usage:

• Always-on continuous recording: Start JFR at JVM launch with maxage and maxsize limits. When an incident occurs, dump the recording.
  Bash

  java -XX:StartFlightRecording=disk=true,maxage=6h,maxsize=1g,dumponexit=true \
       -jar myapp.jar
  

• On-demand recording: Use jcmd to start a recording when investigating an issue:
  Bash

  jcmd <pid> JFR.start name=incident settings=profile duration=120s
  jcmd <pid> JFR.dump name=incident filename=/tmp/incident.jfr
  

• Analysis: Open in JDK Mission Control (JMC) for flame graphs, allocation hot spots, GC analysis, and lock contention visualization.

Why JFR over other profilers:

• No safepoint bias (unlike tools that only sample at safepoints)
• Built into the JVM -- no agent attachment or bytecode instrumentation
• Designed for production use with negligible overhead
• Records system-level events (not just Java) like CPU load, I/O wait

Pre-Java 10 problem: The JVM read /proc/meminfo and /proc/cpuinfo, seeing the host's resources instead of the container's cgroup limits. A container with 2GB limit on a 64GB host would get -Xmx of ~16GB, causing OOM kills.

Java 10+ solution: The JVM reads cgroup v1/v2 limits automatically:

• Runtime.getRuntime().availableProcessors() returns the container's CPU quota
• Default -Xmx is ~25% of the container's memory limit

Key flags for containers:

• -XX:MaxRAMPercentage=75.0 -- Use 75% of container memory for heap (better than -Xmx which is a fixed value)
• -XX:+ExitOnOutOfMemoryError -- Exit on OOM so Kubernetes restarts the pod
• -XX:+HeapDumpOnOutOfMemoryError -- Capture heap dump before exit
• -XX:ActiveProcessorCount=N -- Override detected CPU count if needed
• -XX:+AlwaysPreTouch -- Commit all heap memory at startup so RSS matches committed immediately

Sizing rule: Container memory should be Xmx + 300-500MB to leave room for metaspace, thread stacks, code cache, NIO buffers, and native allocations.

The JVM uses tiered compilation (default since Java 8) that combines two compilers:

• C1 (Client Compiler): Fast compilation with basic optimizations. Used for methods that are warm but not yet hot. Produces code quickly to reduce interpretation overhead.
• C2 (Server Compiler): Slow compilation with aggressive optimizations (inlining, escape analysis, loop unrolling, vectorization). Produces highly optimized native code for the hottest methods.

Compilation tiers: 0 (interpreter) -> 1-3 (C1 with increasing profiling) -> 4 (C2). Methods progress through tiers as they accumulate invocation counts and profiling data.

Key optimizations in C2:

• Inlining: Copies the callee's code into the caller, eliminating method call overhead. The single most impactful optimization.
• Escape Analysis: Determines if an object escapes the method. If not, allocates on the stack (no GC pressure) or eliminates the allocation entirely (scalar replacement).
• Loop Unrolling: Reduces loop overhead by replicating the loop body.
• Deoptimization: If a speculative optimization is invalidated (e.g., a new subclass is loaded), the JVM falls back to the interpreter.

Use -XX:+PrintCompilation to see what is being compiled and -XX:TieredStopAtLevel=1 for fast startup (skip C2).

Capture: Take 3 thread dumps 5-10 seconds apart using jstack <pid> or jcmd <pid> Thread.print. Multiple dumps differentiate stuck threads from merely slow ones.

Analysis steps:

1. Check for deadlocks -- Look for "Found one Java-level deadlock" in the dump output. If present, identify the two (or more) threads and the locks they hold and wait for.

2. Count thread states:

3. Many BLOCKED threads on the same monitor = lock contention bottleneck
4. Many WAITING threads in a thread pool = pool exhaustion (all workers blocked on downstream calls)

5. Threads in RUNNABLE state doing I/O = potential slow external dependency

6. Find stuck threads -- Compare the same thread across all 3 dumps. If it shows the same stack trace in all 3, it is stuck.

7. Follow the chain -- If Thread A is BLOCKED waiting for a lock held by Thread B, check what Thread B is doing. It might be waiting for a database connection, making an HTTP call, or also blocked on another lock.

8. Common root causes:

9. Database connection pool exhaustion (all connections checked out, threads waiting)
10. Downstream service timeout (threads stuck in SocketInputStream.read())
11. Synchronized block contention (one thread holding a lock while doing I/O)
12. Deadlock (circular lock dependency)

Best practice: Always run production JVMs with -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/java/. This automatically captures the evidence you need for post-mortem analysis.

JMH (Java Microbenchmark Harness) is the standard benchmarking framework created by the OpenJDK team. It handles the complexities that make naive benchmarking unreliable:

Why System.currentTimeMillis() fails:

1. JIT warmup: The first N invocations run interpreted or partially compiled. JMH runs configurable warmup iterations to reach steady state.
2. Dead code elimination: If the JIT detects the result is unused, it removes the entire computation. JMH returns results from @Benchmark methods to a Blackhole that prevents elimination.
3. Constant folding: If inputs are compile-time constants, the JIT computes results at compile time. JMH uses @State objects to prevent this.
4. Loop optimization: A tight benchmark loop might be vectorized or eliminated. JMH controls invocation granularity.
5. GC noise: A GC pause during your measurement skews results. JMH uses statistical methods (multiple forks, iterations) to produce reliable averages with error bars.

JMH output includes mean, error margin, confidence intervals, and percentiles -- giving you statistical confidence that one implementation is faster than another.