Work queues, competing consumers and prefetch

Key points

• Many consumers on one queue is the work queue pattern: each message goes to exactly one of them.
• Dispatch is round robin per channel, so a slow consumer is offered the same share as a fast one.
• basic.qos prefetch_count caps unacked deliveries per consumer, and the default 0 means unlimited.
• Unlimited prefetch lets one consumer buffer the whole queue in memory while the others sit idle.
• The global flag is false for a per-consumer limit and true for a per-channel limit.
• Spring AMQP has defaulted prefetch to 250 since 2.0; drop it to 1 only for long, uneven jobs.

Example

# Spring Boot: the listener container settings that matter
spring.rabbitmq.listener.simple.prefetch=20
spring.rabbitmq.listener.simple.concurrency=4
spring.rabbitmq.listener.simple.max-concurrency=10
spring.rabbitmq.listener.simple.acknowledge-mode=auto
spring.rabbitmq.listener.simple.default-requeue-rejected=false

# rule of thumb: prefetch ~= how many messages a consumer can
# finish inside the time it takes the broker to refill the buffer.
# Long jobs -> small prefetch. Tiny fast jobs -> larger prefetch.

Unbounded prefetch means one consumer owns the backlog - always set basic.qos before scaling out.