TTL: message, queue, and per-publish expiration

A startup builds a 'send-an-email-in-30-minutes' feature using TTL+DLX. The retry queue is classic, with `x-message-ttl: 1800000`. Tests pass. In production, a marketing batch publishes 50k messages with per-message expiration='5000' for an urgent flash sale, into the same queue. None of the urgent emails go out for 30 minutes; they sit lazily behind the slow ones. Whose fault is it: the developer, the docs, or the lazy TTL? Trick question: the fix is to know all three flavours of TTL and which queue type evaluates them eagerly.

• **RabbitMQ delayed message exchange plugin** was open-sourced by the core team because the TTL+DLX workaround was so widely used and so easy to misconfigure.
• **MassTransit** (a popular .NET messaging framework) builds its scheduling subsystem on TTL+DLX as the default, and on the delayed-message plugin as an opt-in upgrade.
• **Postmark** publishes engineering posts on using x-expires for per-tenant queues that auto-clean as customers churn out, avoiding a manual cleanup cron.

Предварительные знания

• DLX wiring and dead-letter routing
• Difference between queue arguments and message properties
• Awareness of queue types (classic, quorum)

• Dead Letter Exchanges and DLQ patterns

TTL grew out of AMQP, the delayed plugin out of pain

AMQP 0-9-1 specified per-message `expiration` but no queue-level cap. RabbitMQ added `x-message-ttl` and `x-expires` as proprietary extensions early on (around 2.0) to support the common 'expire after N seconds' pattern without forcing publishers to stamp every message. The TTL+DLX delayed-message trick was discovered by users, not designed by the team; the official delayed-message exchange plugin landed in 3.5 as an explicit response to thousands of community blog posts explaining the workaround.

Three flavours of TTL

RabbitMQ exposes time-to-live at three different scopes, and mixing them up is one of the top sources of weird production behaviour. `x-message-ttl` is a queue argument: every message published into the queue dies after that many milliseconds. The `expiration` property is per-message: the publisher overrides TTL for a single payload, in milliseconds, encoded as a string. `x-expires` is the **queue's own TTL**: an idle queue (no consumers, no publishes) is auto-deleted after that duration.

The `expiration` property must be a **string** of milliseconds. Passing an integer is a common pika/amqplib mistake: the broker silently treats the unknown type as 'no expiration' and your delayed-message pattern stops working. Always `str(ms)` before assigning.

Lazy on classic, eager on quorum

Classic queues check TTL only when a message reaches the head of the queue (lazy expiry). Quorum queues evaluate TTL eagerly, scanning the log periodically and dead-lettering expired entries even if they sit deep in the backlog. The choice matters whenever per-message TTLs vary inside a single queue or when timely DLX routing of stale work is critical.

The lazy-vs-eager distinction is the single biggest reason teams get bitten when migrating retry topologies from classic to quorum (or vice versa). A retry queue that 'worked' on classic because every message had the same TTL behaves very differently when one bad publisher starts sending short-TTL pokes.

If you must keep classic queues but need eager-looking expiry, declare **one queue per TTL tier** (`retry.1s`, `retry.5s`, `retry.30s`). Each queue has uniform TTL, so lazy evaluation does not cause head-of-line blocking.

TTL + DLX = delayed messages

The most popular use of TTL is not actually message lifecycle management; it is delayed delivery. Publish a payload into a buffer queue whose only job is to hold it for the TTL; when TTL fires, the broker dead-letters it into the real work queue. The receiver sees the message exactly when the delay has elapsed. This is how to schedule a reminder or back off a retry without an external scheduler.

For arbitrary per-message delays at large scale, install the official `rabbitmq_delayed_message_exchange` plugin: a new exchange type `x-delayed-message` keeps payloads on disk with a delay header until ready. It avoids the lazy-TTL trap entirely and scales to millions of pending messages.

Delayed-message via TTL+DLX is a transport-level trick, not a real scheduler. It cannot survive broker upgrades that change queue type, it ignores wall-clock skews on restart, and it cannot cancel a scheduled message. For human-facing reminders with strict timing, layer an idempotent scheduler service on top.

x-expires: the queue lifecycle TTL

`x-expires` is a separate concept: how long the **queue object** lives without activity. It is meant for short-lived per-session queues; RPC reply queues, per-WebSocket fan-outs, scratch queues for ephemeral subscribers. It exists to stop your broker from drowning in orphaned per-user queues over time. It does not affect any individual message's TTL.

If you do set `x-expires` on a long-lived queue, make sure your producer's queue_declare runs on every connection (which is idempotent). Otherwise a slow Sunday plus a producer that does not redeclare equals a Monday morning of unbound traffic flowing to the default exchange.

Take it home

• Three TTLs, three scopes: `x-message-ttl` (queue-level), `expiration` (per-message string-ms), `x-expires` (queue lifetime).
• Per-message + queue-level TTL combine as `min(per_message, queue_ttl)`. Smaller value wins.
• Classic queues evaluate TTL lazily at the head; quorum queues evaluate eagerly. Mixed-TTL designs need quorum.
• TTL+DLX is the canonical delayed-delivery hack. Install the delayed-message plugin for arbitrary per-message timing at scale.
• Never set `x-expires` on an infrastructure queue; a quiet weekend will delete it.

Where this leads

TTL is a foundational primitive for retry, scheduling, and per-session topology. The next lesson revisits all of it through the lens of queue types.

• Quorum vs classic vs streams — Queue type changes whether TTL is lazy or eager, and whether DLX even exists
• Acknowledgements: ack, nack, requeue, multiple — TTL-driven dead-letter is the broker-side equivalent of nack(requeue=false)

Связанные уроки

• db-19-redis — Redis EXPIRE provides the same per-key TTL model, useful for comparison.
• sd-07-caching — TTL is a basic cache invalidation strategy, the pattern carries over to queues.
• rmq-10-dlx-dlq — TTL is often combined with DLX to build delayed retry and parking lot patterns.