Adding Backpressure to an Agent Tool Layer

Backpressure becomes necessary when an upstream agent can generate work faster than a downstream dependency can safely consume it. The practical challenge is not merely adding a queue. It is placing a controlled drain in front of the constrained system.

A concrete constrained CRM example

1Agent fleet can generate: 2,000 constrained CRM calls/minute2constrained CRM safe budget: 400 calls/minute

1Without control:2Agent → constrained CRM → 429 → retry → more 429s

The control should live as close as possible to the constrained dependency so unrelated workloads remain healthy.

1Agent Runtime2↓3constrained CRM Task Queue (durable queue)4↓5constrained CRM Worker Pool6↓7Concurrency / rate limiter8↓9constrained CRM API

The queue buffers; the consumer limit creates backpressure

If 2,000 messages are queued and 2,000 workers consume them simultaneously, the dependency is still overloaded. The worker pool must enforce the concurrency or rate budget.

MAX_IN_FLIGHT = 50

1while true:2msg = receive_from_queue()3acquire_crm_slot()4try:5call_crm(msg)6finally:7release_crm_slot()

In a distributed worker fleet the limiter itself must be enforceable across workers, for example through fixed reserved concurrency, partitioned worker pools, or a centralized distributed rate-limit mechanism.

Handle 429s as a signal, not an exception to ignore

• Honor Retry-After when the API provides it.
• Use exponential backoff rather than immediate retries.
• Add jitter so thousands of workers do not retry simultaneously.
• Bound retries; do not retry forever.
• Reduce effective concurrency if 429 rates remain elevated.
• Circuit-break or pause draining when the dependency is clearly unhealthy.

1attempt 1 → 4292wait Retry-After / backoff + jitter3attempt 2 → 4294increase delay5attempt 3 → success

1or:2retry budget exhausted → preserve failure state / escalate

Separate interactive from deferrable traffic

A CRM stage change initiated by a user may be latency-sensitive. A bulk enrichment job usually is not. Treating both identically wastes capacity.

1interactive constrained CRM actions → high-service queue2bulk enrichment → background queue

Scheduler / workers enforce separate budgets

If non-urgent work can run later, deliberately smooth it over time. That is workload shaping: moving throughput to periods where capacity is available.

Know when backpressure becomes admission control

Backpressure regulates flow to a dependency. Admission control decides whether new work should enter the system now. If queue age breaches its SLO, continuing to admit unlimited new work may simply turn an outage into hours of latency.

1New request2↓3Capacity / queue-age check4├─ healthy → accept5├─ saturated but deferrable → queue6└─ cannot meet SLO → reject / ask client to retry

Alarms that make backpressure observable

• queue depth
• oldest-message age
• worker concurrency
• downstream 429 and 5xx rate
• retry count per task
• time spent waiting versus executing
• percentage of work deferred or rejected

Implementation checklist

• Place the queue close to the constrained tool path.
• Enforce a bounded drain rate.
• Respect downstream retry semantics.
• Separate interactive and background budgets.
• Use admission control once latency SLOs can no longer be met.
• Measure queue age, not just queue size.