Stories · Season 2 — Frontrow · Episode 06

The Read Model That Lied

In which the dashboard says one thing, the truth says another, and a bug fixed last month is still poisoning this month's numbers.

The story

Frontrow’s organizer dashboard had a headline number: tickets sold. Venues watched it like a stock ticker. It was fed by a read model — a table updated every time a sale event came through, so the dashboard could answer “how many sold?” instantly without counting the whole event log every time.

One Tuesday, a venue noticed their dashboard said 1,847 sold, but the actual ticket count was 1,851. Small gap. Easy to wave away. Except a week earlier there’d been a bug — since fixed — where refunds were double-counted in the read model. The bug was gone, but the damage it had written was still sitting in the table. The read model had drifted from the truth, and nothing was ever going to bring it back into line on its own.

Mara’s first instinct was to write a careful SQL script to patch the wrong rows. Halfway through she realized she was about to do the thing that caused the problem in the first place: hand-editing a derived table and hoping she got every case right. “Why am I trying to correct this by hand,” she said, “when the correct numbers are already in the event log?”

Why this is hard the traditional way

A read model is a cache of a computation — a convenient summary derived from the real data. The trouble is that in most systems, the read model is treated as the real data. Once a bug writes a wrong value into it, there’s no authority to restore it from. You’re left manually patching a derived table, which is both error-prone and exactly how the next drift gets introduced.

And you usually can’t just “recompute from scratch,” because the inputs — the individual events — were thrown away the moment they updated the summary. The dashboard is the only copy, and the dashboard is wrong.

How Relay changes the ending

In Relay, the event log is the source of truth and projections (read models) are derived from it. That changes the entire posture toward a corrupted read model: you don’t patch it, you rebuild it. Reset the projection, replay the events from the beginning, and it reconstructs itself from the truth — including correctly, now that the bug is fixed.

// Don't hand-edit the wrong rows. Throw the read model away and rebuild it
// from the events that actually happened.
await projectionManager.ResetAsync("organizer-dashboard");
// Replays the full history; the drifted numbers simply cease to exist.

Because the events are still there, the dashboard is disposable — and disposable is a superpower. Fixed a bug in how you compute a number? Rebuild and the past is retroactively correct. Want to do it without downtime? Relay supports blue/green rebuilds: build the corrected read model in the shadows, then switch over when it’s caught up, so organizers never see a half-built dashboard.

What it costs you to ignore this

  • Drift is permanent. A read model that can’t be rebuilt carries every past bug forever.
  • You fix the code but not the data. The bug is gone; its wrong numbers remain, and they’re the ones customers see.
  • Hand-patching breeds the next incident. Manually editing derived tables is how drift gets introduced, not cured.
  • You can’t evolve your views. Without rebuilds, changing how a metric is computed means it’s only right going forward — the past stays wrong.