Outbound HTTP: pools, timeouts and keep-alive

Key points

• The single most important setting is a read timeout. Several client stacks default to waiting indefinitely, so one unresponsive downstream parks every request thread and the outage propagates upstream.
• Two timeouts matter and they are different: connect bounds establishing the socket, read bounds waiting for bytes once connected. A downstream that accepts and then stalls is only caught by the second.
• Connection pools are per client and often per route. A default of a handful of connections per host becomes the real concurrency limit no matter how many threads you have.
• Reusing connections matters more than it looks. A fresh TLS handshake costs a round trip or two, so a pool that keeps connections alive can halve the latency of a small request.
• Give each downstream its own client and pool. Sharing one pool across dependencies means a slow service can consume every connection and starve the others - the bulkhead argument, applied to HTTP.
• Set the pool acquisition timeout too. Without it, exhaustion shows up as unbounded waiting rather than a fast, attributable failure.

Example

@Configuration
class HttpClientConfig {

    // One client and one pool PER downstream, so a slow dependency cannot
    // consume the connections another one needs.
    @Bean
    RestClient paymentsClient(RestClient.Builder builder) {
        PoolingHttpClientConnectionManager pool = PoolingHttpClientConnectionManagerBuilder
                .create()
                .setMaxConnTotal(50)
                .setMaxConnPerRoute(50)      // the default is far lower - usually the real cap
                .build();

        CloseableHttpClient http = HttpClients.custom()
                .setConnectionManager(pool)
                .setDefaultRequestConfig(RequestConfig.custom()
                        .setConnectTimeout(Timeout.ofSeconds(2))              // socket setup
                        .setResponseTimeout(Timeout.ofSeconds(5))             // waiting for bytes
                        .setConnectionRequestTimeout(Timeout.ofSeconds(1))    // waiting for a slot
                        .build())
                .evictIdleConnections(TimeValue.ofSeconds(30))
                .build();

        return builder
                .baseUrl("https://payments.internal")
                .requestFactory(new HttpComponentsClientHttpRequestFactory(http))
                .build();
    }
}

/*
 * The failure this prevents:
 *
 *   no read timeout   -> threads park forever on a stalled downstream
 *                        -> the pool empties -> this service stops responding
 *                        -> its callers time out -> the outage spreads upstream
 *
 * A read timeout converts an unbounded hang into a fast, attributable error
 * that a circuit breaker can then act on.
 */

Set connect, read and pool-acquisition timeouts on every client, and give each downstream its own pool.