Virtual threads in a real service

Key points

• Virtual threads raise the concurrency ceiling of blocking code. They do not make any single request faster - one call takes exactly as long as before.
• They only help when threads are the constraint. If the bottleneck is CPU, a database pool, or a downstream rate limit, more in-flight requests just move the queue.
• Pinning was the trap before JDK 24: a virtual thread inside a synchronized block could not unmount, so it held its carrier thread while blocked. A `ReentrantLock` never pins.
• Pool sizing inverts. You no longer size a thread pool for concurrency - you use an unbounded virtual executor and instead bound the real resources with a semaphore or a connection pool.
• ThreadLocal still works but costs more at this scale: a million virtual threads each holding a ThreadLocal is a million objects. ScopedValue exists for exactly that reason.
• Do not pool virtual threads. They are cheap to create and pooling reintroduces the very limit you removed - one per task is the intended model.

Example

# Spring Boot 3.2+ on Java 21 - one property.
spring:
  threads:
    virtual:
      enabled: true

# Find pinning before it finds you (needed below JDK 24):
#   java -Djdk.tracePinnedThreads=full -jar app.jar
#
#   Thread[#42,VirtualThread]... <pinned>
#     java.base/java.io.PrintStream.write   <- synchronized inside

---
// WRONG - pooling virtual threads recreates the limit you removed.
var wrong = Executors.newFixedThreadPool(200, Thread.ofVirtual().factory());

// RIGHT - one virtual thread per task, unbounded.
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (var task : tasks) { executor.submit(task); }
}   // close() waits for every task

// Bound the REAL resource instead of the thread count.
private final Semaphore downstreamLimit = new Semaphore(50);

Result call(Request r) throws InterruptedException {
    downstreamLimit.acquire();       // 10,000 virtual threads, 50 concurrent calls
    try {
        return client.send(r);
    } finally {
        downstreamLimit.release();
    }
}

// Pinning: replace synchronized with a lock on any path that blocks.
// BEFORE - pins the carrier thread across the I/O
synchronized (this) { return remoteCall(); }

// AFTER - unmounts correctly
private final ReentrantLock lock = new ReentrantLock();
lock.lock();
try { return remoteCall(); } finally { lock.unlock(); }

/*
 * WHAT CHANGES AND WHAT DOES NOT:
 *
 *   concurrent blocking requests   200 -> tens of thousands   CHANGES
 *   latency of one request         identical                  no
 *   database pool size             identical                  no
 *   downstream rate limits         identical                  no
 *   need for timeouts              unchanged, and more urgent no
 */

Never pool virtual threads - bound the real resource with a semaphore, and replace synchronized with a lock on blocking paths.