Tuning GC from evidence, not folklore

Key points

• Almost every GC problem is one of three things: the live set is too large for the heap, the allocation rate is too high, or the collector is mismatched to the latency requirement.
• Measure the live set from the heap occupancy after a full collection, not from total usage. A heap that is 90% full but 10% live is fine; one that is 60% full and 55% live is in trouble.
• Allocation rate is the usual real culprit. Cutting garbage - reusing buffers, avoiding boxing in hot paths, not building strings that are discarded - beats every flag you could set.
• Collector choice follows the requirement, not fashion. G1 balances throughput and pause. ZGC and Shenandoah target sub-millisecond pauses at some throughput cost. Parallel still wins on pure throughput for batch work.
• Sizing the heap larger reduces collection frequency but increases the work per collection for most collectors, so it trades pause count against pause length rather than removing the problem.
• Never tune from a single symptom. Turn on GC logging, look at the distribution of pauses and the occupancy trend over hours, and change one flag at a time.

Example

# Always log. This costs almost nothing and is the only real evidence.
-Xlog:gc*,gc+heap=debug,safepoint:file=gc.log:time,uptime,level,tags:filecount=5,filesize=20M

# The three numbers to extract:
#
#  1. LIVE SET     heap used immediately AFTER a full GC
#     rising over hours -> a leak, and no flag will save you
#
#  2. ALLOCATION RATE   MB/s of new objects
#     > ~1 GB/s on a normal service -> fix the code, not the collector
#
#  3. PAUSE DISTRIBUTION   p50 / p99 / max, not the average
#     the max is what your users actually felt

# Sensible G1 starting point - set the goal, not the mechanism.
-XX:+UseG1GC
-XX:MaxGCPauseMillis=200          # a TARGET; G1 sizes regions to meet it
-Xms4g -Xmx4g                     # equal: avoid resize pauses and surprises

# Latency-critical, and willing to pay throughput for it:
-XX:+UseZGC -XX:+ZGenerational    # sub-millisecond pauses, more CPU

# Batch throughput, pauses irrelevant:
-XX:+UseParallelGC

# In a container, let the JVM see the real limits:
-XX:MaxRAMPercentage=70           # NOT -Xmx guessed from the host size

# Anti-patterns worth naming:
#   -XX:+UseConcMarkSweepGC   removed in JDK 14
#   System.gc()               a full pause on demand; disable with
#                             -XX:+DisableExplicitGC
#   copying flags from a blog without measuring first

Get live set, allocation rate and the pause distribution before touching a flag - most GC problems are allocation problems in disguise.