The Email That Sent Itself Five Times

The story

The outbox from the last story fixed the missing pickles. The warehouse now reliably received every OrderPaid message. Reliability achieved! Dana added a handler that, on OrderPaid, emailed the customer a receipt and charged a small handling fee.

A week later, a regular customer — kindly, but with the tone of someone who has been wronged — forwarded five identical receipts for a single jar of cornichons. Five emails. And, far worse, five handling-fee charges on their card.

Dana stared at the logs in disbelief. The order was placed once. The message was published once. So why did the handler run five times?

Because message brokers, like the post office, promise to deliver your message — they just don’t promise to deliver it exactly once. If the broker isn’t 100% sure the handler finished (a network blip, a slow ack, a redeploy mid-message), it does the safe thing and delivers again. “At least once” is the honest default of every reliable broker on earth. Usually it’s once. Sometimes it’s five.

Why this is hard the traditional way

Here’s the uncomfortable truth: reliable delivery and duplicate delivery are the same feature. The only way a broker can guarantee it never loses your message is to be willing to send it again when unsure. You cannot have at-least-once reliability and a guarantee of exactly-once delivery from the broker. Physics, networks, and computer science all agree.

So duplicates aren’t a bug you can fix in the broker. They’re a fact you have to handle in the consumer. And handling them by hand is fiddly: you need to remember every message you’ve already processed, check that memory before doing anything with a side effect, and update it atomically — or you’ll have two copies of the handler racing, both deciding “nope, haven’t seen this one,” both charging the card. The race condition that bills your customer twice is depressingly easy to write.

How Relay changes the ending

Relay puts an inbox in front of message consumers. Before a message is processed, Relay records its ID in an inbox table. If that ID has been seen before, the message is acknowledged and dropped — the handler simply doesn’t run again. The check and the processing happen atomically, so two racing copies can’t both slip through.

// You write the handler as if every message arrives exactly once.
// Relay's inbox makes that fiction true — duplicates are de-duplicated for you.
public sealed class SendReceiptHandler : IMessageConsumer<OrderPaid>
{
    public async Task Consume(OrderPaid msg, CancellationToken ct)
    {
        await _email.SendReceiptAsync(msg.OrderId);  // runs once, even if delivered five times
        await _billing.ChargeHandlingFeeAsync(msg.OrderId);
    }
}

The customer gets one receipt and one charge, no matter how many times the broker gets nervous and redelivers. Dana didn’t have to write a single line of dedup-tracking logic, and — crucially — didn’t have to get the race condition right by hand.

(The customer was refunded four handling fees and a sincere apology. Pemberton & Crumb threw in a free jar of cornichons. Relations were restored.)

What it costs you to ignore this

• Side effects multiply. Duplicate emails are embarrassing; duplicate charges, shipments, or payouts are real money and real trust, gone.
• It’s intermittent and unreproducible. Duplicates only happen during blips and redeploys, so it passes every test and fails in production.
• Hand-rolled dedup hides race conditions. The “have I seen this?” check is exactly the kind of code that looks right and bills people twice under load.
• It scares teams away from retries. Fearing duplicates, teams disable the retries that make messaging reliable in the first place — trading one problem for the worse one from story 003.