When the JIT gives up: deoptimisation

Key points

• The JIT optimises speculatively using what it has observed so far. If only one implementation of an interface has been seen, it inlines that one directly and skips the virtual dispatch.
• The speculation is guarded. When a second implementation finally appears, the guard fails and the method is deoptimised - thrown away, execution falls back to the interpreter, and it is recompiled more conservatively.
• This is why a method can get slower after running correctly for hours. Nothing changed in the code; the shape of the data changed.
• Monomorphic call sites (one type) inline. Bimorphic (two) still can. Megamorphic (many) fall back to a real virtual call and are much harder to optimise.
• Uncommon traps are the same mechanism: a branch never taken is not compiled at all. The first time it is taken, the method deoptimises and recompiles - which shows up as a latency spike on a rare path.
• The practical implication is about benchmarking. A microbenchmark that feeds one type gets an unrealistically inlined result; production, with many types, does not. This is exactly why JMH exists.

Example

# See what the JIT is actually doing.
-XX:+UnlockDiagnosticVMOptions
-XX:+PrintCompilation
-XX:+PrintInlining

# Output worth recognising:
#
#   1234  456  %  4  com.example.Service::process @ 12 (86 bytes)
#                ^     ^
#                |     tier 4 = C2, fully optimised
#                on-stack replacement (a long-running loop)
#
#   1250  456     4  com.example.Service::process  MADE NOT ENTRANT
#                                                  ^ DEOPTIMISED here
#
# "made not entrant" after a long healthy period = a speculation just
# broke. Usually a second implementation of an interface appeared.

---

/*
 * CALL SITE SHAPES:
 *
 *   monomorphic  1 type seen   -> inlined, effectively a direct call
 *   bimorphic    2 types       -> both inlined behind a type check
 *   megamorphic  3+ types      -> real virtual dispatch, little inlining
 *
 * The trap this sets for benchmarks:
 *
 *   for (int i = 0; i < N; i++) { handler.handle(x); }
 *
 *   Benchmark passes ONE implementation -> monomorphic, inlined, fast.
 *   Production has SIX                  -> megamorphic, and the number
 *                                          you published is meaningless.
 *
 * Use JMH, and feed it the type mix production actually has.
 */

The JIT bets on what it has seen; when the bet breaks the method is discarded and recompiled, which is why code can slow down without changing.