🌿At-least-once delivery makes deduplication your job
Most message systems promise at-least-once delivery: your handler will see every message - and some of them twice. "Exactly-once delivery" end to end is largely a marketing term; the instant a network or a crash sits between producer and consumer, redelivery is unavoidable.
So the duplicate is not an edge case. It's the contract.
The consumer must produce the same result when it processes the same message twice - via an idempotency key, a dedup table keyed on message id, or a naturally idempotent operation. Teams that treat duplicates as "shouldn't happen" ship double-charged customers and double-sent emails, then discover the guarantee they relied on never existed.
Design every consumer as if each message arrives twice, because eventually one will.
Related
Linked from
- 🌿 Distributed Systems
An evolving map of hardwon lessons about building systems that span more than one machine…
- 🌳 Idempotency is a property, not a mechanism
Idempotency describes an outcome applying an operation N times equals applying it once.…
- 🌿 A shared database is distributed coupling in disguise
Two services that write to the same table are not two services. They're one system with a…
- 🌿 The outbox pattern gets you at-least-once without a shared database
The trap is the dual write: your service updates its database and publishes an event to a…