Caching between services

Key points

• Cache the call you actually repeat. A cache over a rarely-read row saves nothing and can still hand you a stale answer.
• TTL is the honest control: it states how old a reply may be. A short TTL usually beats clever invalidation logic.
• A local Caffeine cache is fastest, but every pod holds its own copy. Redis is shared between pods, at one network hop.
• Guard the stampede: when a hot key expires, every pod calls the origin at once. Jitter the TTL, refresh ahead, or lock per key.
• Never cache what must not be served twice, such as a one-time token, and never cache one user under a key another user can hit.

Example

import java.util.HashMap;
import java.util.Map;

/** A read-through TTL cache with an injected clock, so expiry is visible
 *  without the demo having to wait for real time to pass. */
public class Main {

    static final class TtlCache {
        private record Entry(String value, long expiresAt) {}

        private final Map<String, Entry> map = new HashMap<>();
        private final long ttlMillis;
        int originCalls;

        TtlCache(long ttlMillis) { this.ttlMillis = ttlMillis; }

        String get(String key, long now) {
            Entry hit = map.get(key);
            if (hit != null && now < hit.expiresAt()) {
                return hit.value();                  // served from the copy
            }
            originCalls++;                           // miss: ask the owner
            String fresh = "stock-of-" + key + "@" + now;
            map.put(key, new Entry(fresh, now + ttlMillis));
            return fresh;
        }
    }

    public static void main(String[] args) {
        TtlCache cache = new TtlCache(5000);

        System.out.println(cache.get("JCH-9", 0));       // miss  -> origin
        System.out.println(cache.get("JCH-9", 1000));    // hit
        System.out.println(cache.get("JCH-9", 4999));    // hit, still fresh
        System.out.println(cache.get("JCH-9", 5000));    // TTL up -> origin

        System.out.println("calls to the origin: " + cache.originCalls);
    }
}

Every cache trades freshness for speed, so set the TTL to the staleness you can defend.