Backpressure Patterns — Flow Control for Resilient Distributed Systems

What is backpressure?#

Backpressure is a flow-control mechanism where a slow consumer signals a fast producer to slow down. Without it, the producer floods the consumer, buffers grow unbounded, memory runs out, and the system crashes.

The term comes from fluid dynamics: pressure that opposes the desired flow. In software, it means the same thing — downstream resistance that propagates upstream.

Why backpressure matters#

Consider a data pipeline: an API ingests 10,000 events per second, a message queue buffers them, and a database consumer writes at 2,000 per second. Without backpressure:

1. The queue grows by 8,000 messages per second
2. After 10 minutes, 4.8 million messages are buffered
3. The broker runs out of memory or disk
4. The entire pipeline fails

Backpressure prevents this by making the producer aware of the consumer's capacity.

Push-based vs pull-based models#

Push-based (no backpressure)#

The producer sends data whenever it is ready. The consumer must handle whatever arrives:

• TCP without flow control
• Fire-and-forget message publishing
• Event emitters with no subscriber feedback

If the consumer is slower than the producer, data is lost or buffers overflow.

Pull-based (natural backpressure)#

The consumer requests data when it is ready. The producer only sends what was requested:

• HTTP request/response
• Kafka consumer polling
• Iterator/generator patterns
• Reactive Streams request(n)

The consumer controls the flow rate. Backpressure is built into the protocol.

Hybrid: push with backpressure signals#

The producer pushes data but respects signals from the consumer:

• TCP flow control (window size)
• Reactive Streams with request(n) demand signaling
• gRPC flow control
• WebSocket with drain events

This combines the low-latency of push with the safety of pull.

The Reactive Streams specification#

The Reactive Streams spec (adopted into Java 9 as java.util.concurrent.Flow) defines four interfaces:

• Publisher: produces elements
• Subscriber: consumes elements
• Subscription: the link between them, with request(n) and cancel()
• Processor: both a subscriber and a publisher (a transformation stage)

The protocol:

1. Subscriber calls subscribe() on Publisher
2. Publisher calls onSubscribe(subscription) on Subscriber
3. Subscriber calls subscription.request(n) to demand n elements
4. Publisher calls onNext() up to n times
5. Subscriber requests more when ready

No element is sent without demand. This is non-blocking backpressure.

Queue-based buffering strategies#

When producers and consumers run at different speeds, queues absorb bursts. But queues need bounds:

Bounded queues#

Set a maximum size. When full, choose a strategy:

Unbounded queues (avoid these)#

Unbounded queues defer the problem. They grow until the system runs out of memory. Always set bounds, even if they are generous.

Priority queues#

When shedding load, shed low-priority work first. Priority queues let critical requests (health checks, payments) proceed while bulk operations (analytics, reports) are deferred.

Load shedding#

When backpressure is not enough — when the system is at capacity and cannot slow producers — load shedding deliberately drops work:

• Random early detection (RED): as queue utilization rises, randomly drop an increasing percentage of incoming requests
• CoDel (Controlled Delay): drop packets that have been in the queue longer than a target latency
• Priority-based shedding: drop low-priority requests first
• Admission control: reject requests at the entry point before they consume internal resources

HTTP load shedding#

Return 503 Service Unavailable with a Retry-After header. Well-behaved clients back off. This is more graceful than letting the server crash under load.

Kafka consumer lag#

If a Kafka consumer falls behind:

1. Scale out consumers (add partitions and consumer instances)
2. Skip to the latest offset and backfill from a snapshot
3. Route lagging partitions to a catch-up consumer group

Circuit breaker integration#

Backpressure and circuit breakers complement each other:

• Backpressure slows the flow when a downstream service is slow
• Circuit breaker stops the flow when a downstream service is failing

Together:

1. Consumer slows down (backpressure)
2. Latency increases beyond threshold (circuit breaker opens)
3. Requests are immediately rejected or routed to a fallback
4. After a timeout, the circuit breaker half-opens and tests with a few requests
5. If the downstream service recovers, normal flow resumes

This prevents cascading failures: without the circuit breaker, backpressure alone would cause upstream services to accumulate blocked threads and eventually fail too.

Tools and frameworks#

RxJava#

RxJava 2+ supports backpressure through the Flowable type (as opposed to Observable, which does not support backpressure):

• onBackpressureBuffer() — buffer with optional bounds
• onBackpressureDrop() — discard items when downstream is slow
• onBackpressureLatest() — keep only the most recent item

Project Reactor#

The reactive library for Spring WebFlux. Flux supports backpressure natively:

• limitRate(n) — prefetch n elements at a time
• onBackpressureBuffer(maxSize, dropStrategy)
• onBackpressureDrop()
• Integrates with Spring WebFlux, R2DBC, and reactive MongoDB drivers

Akka Streams#

Akka Streams implements Reactive Streams with a graph-based DSL:

• Every stage has automatic backpressure
• buffer(size, OverflowStrategy) — explicit buffering with overflow control
• throttle(elements, duration) — rate limiting
• Fusing: adjacent stages run on the same thread to reduce overhead
• Async boundaries: stages on different threads communicate via backpressured message passing

Kafka#

Kafka provides natural backpressure through pull-based consumption:

• Consumers poll at their own pace (max.poll.records controls batch size)
• Producers can be throttled by broker quotas
• Consumer lag metrics signal when consumers fall behind

gRPC#

gRPC has built-in flow control at the HTTP/2 layer:

• Per-stream and per-connection flow control windows
• Servers can delay responses to signal backpressure
• Client-side streaming respects server readiness

Backpressure anti-patterns#

1. Unbounded queues everywhere — defers failure, does not prevent it
2. Ignoring consumer lag — if you do not monitor it, you do not know you have a problem
3. Retrying without backoff — amplifies load on an already-stressed system
4. Blocking the event loop — in async systems, blocking defeats the purpose of non-blocking backpressure
5. Shedding load too late — by the time the system is overloaded, shedding may not recover it

Monitoring backpressure#

Key metrics to watch:

• Queue depth: how many messages are buffered
• Consumer lag: how far behind the consumer is (Kafka, event streams)
• Processing latency: time from enqueue to dequeue
• Rejection rate: how many requests are being shed
• Thread pool saturation: how many threads are blocked waiting

Alert when queue depth or consumer lag grows steadily — it means the consumer cannot keep up.

Visualize backpressure in your architecture#

On Codelit, generate a streaming data pipeline or microservices architecture to see where backpressure applies. Click on queues and service boundaries to explore buffering strategies and flow control.

---

This is article #230 in the Codelit engineering blog series.

Design resilient architectures visually at codelit.io.