Timeouts, retries and circuit breakers

Key points

• Every remote call needs a timeout. Without one, a slow dependency holds your threads until the pool is empty.
• Retry only idempotent calls, a small number of times, with exponential backoff and jitter.
• Retrying a request that already succeeded but timed out is how one order becomes three.
• A circuit breaker counts recent failures. Past the threshold it opens and fails calls instantly.
• After a wait it moves to HALF_OPEN and lets a few probes through: success closes it, failure opens it again.
• Give the breaker a fallback. A slightly stale cached answer beats a 500 for almost every screen.

Example

@Service
class RateService {

    @CircuitBreaker(name = "rates", fallbackMethod = "cachedRate")
    @Retry(name = "rates")                 // idempotent GET, so retrying is safe
    public Rate lookup(String currency) {
        return ratesApi.byCurrency(currency);
    }

    // Same signature plus the exception. Called when the breaker is open or retries ran out.
    Rate cachedRate(String currency, Throwable cause) {
        log.warn("rates degraded: {}", cause.toString());
        return lastKnown.getOrDefault(currency, Rate.parity(currency));
    }
}

// application.yml
// resilience4j:
//   circuitbreaker.instances.rates:
//     slidingWindowSize: 20
//     failureRateThreshold: 50          # percent
//     waitDurationInOpenState: 10s
//     permittedNumberOfCallsInHalfOpenState: 3
//   retry.instances.rates:
//     maxAttempts: 3
//     waitDuration: 200ms
//     enableExponentialBackoff: true
//     exponentialBackoffMultiplier: 2

Timeouts stop the bleeding, retries handle a blip, and a breaker stops you hammering something already down.