Articles · Event Sourcing ·006

Concurrency & Snapshots

Runnable sample on GitHub

Sample: samples/006-concurrency-and-snapshots — the ledger with optimistic concurrency and snapshots. dotnet test (needs Docker).

Prerequisite: 005 — Event Sourcing Basics.

Overview

Article 005’s event sourcing works beautifully for a fresh account with three events. Production breaks that idyll two ways. First, two requests change the same aggregate at once — without a referee, one silently overwrites the other (a lost update). Second, streams grow — replaying fifty thousand events on every load is slow. Relay’s answers are optimistic concurrency (detect concurrent writes via an expected-version check) and snapshots (periodically persist state so loads replay only the tail). Neither changes your domain code; both are configuration plus, for snapshots, a small interface.

Why this exists

Concurrency. Imagine two ATMs withdrawing from one account simultaneously. Both load balance 100, both check “100 ≥ 80,” both append a withdrawal. Without a concurrency check, the account ends at 20 having dispensed 160 — the second write overwrote the first’s effect on the version, and the invariant was evaluated against stale state. This is the lost update problem, and it is not theoretical: any system with concurrent users hits it. You must either prevent concurrent writes (locks) or detect them (versions). Event sourcing detects them, cheaply, because every append already declares which version it expected to follow.

Snapshots. Replaying events is linear in stream length. For most aggregates that is forever fine — an order has dozens of events, an account hundreds. But a long-lived, high-traffic aggregate (a busy account over years, a long-running process) can accumulate tens of thousands of events, and replaying them all on every load becomes the dominant cost. A snapshot is a periodically-saved copy of the aggregate’s state at a version, so loading means “restore the snapshot, then replay only the events since” — turning an O(n) load into an O(events-since-last-snapshot) one.

When to use this

Optimistic concurrency is not optional — it is on for every event-sourced aggregate, because the expected-version check is how appends work. The decision is how you handle a conflict (retry, or surface a 409), not whether to detect it. Use it always; it is free.

Snapshots, by contrast, are a targeted optimization. Turn them on when:

  • A stream is long and hot. A ledger account with years of transactions, a sensor’s reading history, a subscription with thousands of billing events — anything where the replay cost on load is measurably hurting latency.
  • Loads dominate and streams keep growing. If an aggregate is read constantly and its stream never stops growing, replay cost grows unbounded; snapshots cap it.
  • You have measured the problem. The honest trigger is a profiler or a latency graph showing replay as the cost, not a hunch.

Concrete domains: a payments ledger (long-lived accounts), inventory for a high-velocity SKU (thousands of reservations), a logistics shipment with a long, eventful journey, or a booking resource reserved and released thousands of times.

When not to use this

Don’t add snapshots when:

  • Streams are short. An order with 20 events replays in microseconds; a snapshot saves nothing and adds a table, a serialization format, and a thing to reason about. This is the common case — most aggregates never need snapshots.
  • You haven’t measured. Premature snapshotting is premature optimization with extra steps. Replay is cheap until it isn’t; prove it isn’t before adding the machinery.
  • The aggregate is write-heavy and read-light. Snapshots optimize loads. If you rarely load (you mostly append), they buy little and still cost a write each cadence.

The costs of snapshots: another serialized representation to maintain (and version, though more forgivingly than events — see below); a SnapshotEvery knob to tune; and the subtle trap of treating the snapshot as authoritative when it is only a cache. For optimistic concurrency the “cost” is different — you must decide what to do on conflict, and a naive retry loop can mask a real contention problem.

Concepts

Expected version. Every aggregate tracks the version it was loaded at (-1 for brand new). When the transaction appends its new events, it asserts that version. The event store enforces it with a unique index on (aggregate_id, version): if another transaction already wrote the next version, the insert collides and Relay throws ConcurrencyConflictException. The conflicting command rolls back entirely — no partial writes.

Handling a conflict. Two reasonable responses: retry (reload the aggregate, re-apply the intent against fresh state, append again) for blind operations like “deposit 10”; or surface it (map to HTTP 409, article 004) for operations the user must reconsider against new state. Retry suits commutative/blind changes; surfacing suits decisions (“you edited a stale version”).

Snapshot. An aggregate implements ISnapshotable: GetSnapshotState() returns a serializable state object; RestoreFromSnapshot(state, version) rebuilds from it. With SnapshotEvery = N, a command that advances the aggregate across a multiple of N writes a snapshot at that version, in the same transaction as the events. KeepSnapshots (default 2) prunes old snapshots, keeping the newest as primary and the previous as a fallback.

Snapshots are a cache, not a source of truth. The events remain authoritative. If a snapshot is missing or fails to deserialize, the repository silently falls back to a full replay from version 0. This is what makes snapshots low-risk: get the snapshot format wrong and you lose performance, not correctness.

Architecture

graph TD
    subgraph Concurrency
        A1[Append events, expectedVersion = loaded version] --> U{unique aggregate_id, version?}
        U -->|ok| C1[COMMIT]
        U -->|collision| X[ConcurrencyConflictException → ROLLBACK]
    end
    subgraph "Load with snapshots"
        L1[GetByIdAsync] --> S{latest snapshot?}
        S -->|none / corrupt| R0[replay from version 0]
        S -->|version k| R1[RestoreFromSnapshot k, then replay events > k]
        R0 --> Agg[rehydrated aggregate]
        R1 --> Agg
    end

Building it step by step

The sample is samples/006-concurrency-and-snapshots.

1. Make the aggregate snapshotable

public sealed record AccountSnapshot(Guid Id, string Owner, decimal Balance);

public sealed class Account : AggregateRoot<Guid>, ISnapshotable
{
    // … events, command methods, ApplyEvent as in article 005 …
    public object GetSnapshotState() => new AccountSnapshot(Id, Owner, Balance);

    public void RestoreFromSnapshot(object state, long version)
    {
        var s = ((JObject)state).ToObject<AccountSnapshot>()!; // the repo hands back a Newtonsoft JObject
        SetId(s.Id); Owner = s.Owner; Balance = s.Balance;
        Version = version;
    }
}

GetSnapshotState returns whatever captures the aggregate’s state; RestoreFromSnapshot is its inverse. Keep it simple — it is a cache entry, and a broken one just triggers a full replay.

2. Configure the cadence

services.AddRelayEventStoreEfCore<LedgerDbContext>(s => s.SnapshotEvery = 2); // tune to your stream length

That is the entire opt-in. KeepSnapshots defaults to 2; raise SnapshotEvery for longer streams (a snapshot every few hundred events is typical) so you snapshot rarely but still bound replay.

3. Concurrency is already on

Nothing to configure — the expected-version check is intrinsic to appending. You only decide how to react to a ConcurrencyConflictException: catch-and-retry, or let it map to a 409.

Complete source code

File Contents
Accounts/Account.cs The account + ISnapshotable + an AppliedFromHistory counter
LedgerFixture.cs AddRelayEventStoreEfCore(s => s.SnapshotEvery = 2)
ConcurrencyAndSnapshotTests.cs The conflict + the snapshot cadence/replay-skip

Running the example

dotnet test samples/006-concurrency-and-snapshots/Ledger.Snapshots.Tests   # needs Docker

Testing

A stale write is rejected and rolls back. The deterministic way to force a conflict is to append a version the store already has — opening the same account twice does exactly that:

await Bus.Execute<OpenAccountCommand, Account>(new OpenAccountCommand(id, "first"), default);
var act = () => Bus.Execute<OpenAccountCommand, Account>(new OpenAccountCommand(id, "duplicate"), default);
await act.Should().ThrowAsync<ConcurrencyConflictException>();
(await eventStore.GetEventsAsync(id, 0, default)).Should().HaveCount(1); // the conflict rolled back

Two genuinely concurrent writers are detected by the same check — both load version k, both try to append k+1, the unique index lets exactly one through. The duplicate-open is just the deterministic way to demonstrate it in a test (a real race is timing-dependent and flaky to assert).

A snapshot bounds replay. The AppliedFromHistory counter (incremented in ApplyEvent) makes “replayed nothing” observable:

// SnapshotEvery = 2; open + two deposits ⇒ a snapshot at version 2 (the head)
var snapshot = await ctx.Set<SnapshotRecord>().OrderByDescending(s => s.Version).FirstOrDefaultAsync(...);
snapshot!.Version.Should().Be(2);

var loaded = await repository.GetByIdAsync(id, default);
loaded!.Balance.Should().Be(15m);
loaded.AppliedFromHistory.Should().Be(0, "the head snapshot covers every event, so none replay");

This is tested against real PostgreSQL because the unique-index conflict and the atomic snapshot+events commit are database behaviors — a mock would prove nothing.

Production considerations

  • Pick SnapshotEvery from stream growth, not vibes. If accounts gain ~1000 events/year and you want loads to replay ≤ a few hundred, snapshot every few hundred events. Too frequent wastes writes; too rare lets replay creep back up. Measure, then set.
  • Decide your conflict policy per operation. Blind, commutative commands (deposit, append-only) are good retry candidates — reload, re-apply, re-append, with a small bounded retry and jitter. Decisions made against displayed state (edit this record) should surface the conflict (409) so the user re-decides; silently retrying would re-apply their stale intent.
  • Bound and back off retries. A retry loop with no cap turns a hot-aggregate contention problem into a CPU fire. If you are retrying often, the real fix is usually a smaller aggregate or a different workflow (a saga, article 015), not more retries.
  • Snapshot schema evolution is forgiving — but not free. Because a snapshot that won’t deserialize triggers a full replay, you can change the snapshot shape and old snapshots simply get ignored (replay rebuilds, then the next cadence writes a new-shape snapshot). That is a feature, but a deploy that invalidates every snapshot means every aggregate does one full replay first — fine for short streams, a thundering-herd risk for very long ones, so consider rebuilding snapshots proactively.
  • The contention is a design signal. Frequent conflicts on one aggregate usually mean the boundary is too coarse (everyone writes the same aggregate). Splitting it, or moving coordination into a saga, often beats fighting the conflicts.

Common mistakes

  • Treating the snapshot as the source of truth. It is a cache. Never delete events because “we have a snapshot,” never trust a snapshot you can’t reproduce by replay. The events are authoritative; the snapshot is disposable.
  • Snapshotting short streams. Adding snapshots to an aggregate with 20 events optimizes nothing and adds a serialization format to maintain. Don’t.
  • Swallowing ConcurrencyConflictException. Catching it and returning success re-introduces the lost-update bug you were detecting. Either retry deliberately or surface it; never ignore it.
  • Unbounded retry loops. Retrying a conflict forever under contention melts CPU and hides the real problem. Cap retries, add backoff/jitter, and treat persistent conflicts as a modelling smell.
  • A RestoreFromSnapshot that disagrees with ApplyEvent. If restoring from a snapshot produces different state than replaying the events to the same version, you have a correctness bug that only bites when snapshots are present. Keep the snapshot state a faithful capture of what replay produces.

Tradeoffs

Benefits. Concurrency conflicts are detected rather than silently lost, with no held locks (scales better than pessimistic locking); long streams load in bounded time; and snapshots are low-risk because they degrade to replay, never to incorrectness.

Costs. You must design a conflict-handling policy (and resist naive retry loops); snapshots add a serialized representation, a cadence to tune, and the temptation to over-trust them; and a snapshot-invalidating deploy can cause a replay stampede for very long streams.

Alternatives

  • Pessimistic locking (SELECT … FOR UPDATE / app locks). Prevent concurrency by serializing access to an aggregate. Conceptually simple, but it holds locks across the operation (hurting throughput and risking deadlocks) and does not fit an append-only store as naturally as version checks. Optimistic concurrency is the better default for event sourcing; reach for locks only for genuinely high-contention, must-not-conflict sections. For exactly that narrow case Relay ships IAggregateWriteLock (AddRelayAggregateWriteLock): take it inside the command before loading the aggregate (AcquireAsync(aggregateType, id)) and contending writers to the same id serialize on the unit-of-work transaction (released at commit/rollback) instead of racing and retrying — other aggregate instances are unaffected.
  • Last-write-wins (no concurrency control). “Whoever commits last wins.” Trivial and wrong for anything with invariants — it is the lost-update bug as a policy. Acceptable only for truly conflict-free data (e.g. independent fields), which an aggregate, by definition, is not.
  • No snapshots, ever. Perfectly correct, and the right choice until a stream is long enough to hurt. The alternative to snapshots is simply not having them — and for most aggregates that is the answer.

Lessons from production systems

  • Most aggregates never need snapshots, and teams add them too early. The reliable pattern is: ship without snapshots, watch load latency, add them to the specific aggregates whose streams actually grow unbounded. A blanket “snapshot everything” policy is complexity with no measured payoff.
  • The conflict-handling decision is where bugs hide. Teams wire up the detection (it’s automatic) and then either swallow conflicts (lost updates return) or retry blindly (contention storms). Decide, per command, between “retry the intent” and “tell the user,” and write that policy down.
  • Frequent conflicts are a modelling message. The aggregates that conflict constantly are almost always too big — a single Account everyone hammers, a god aggregate. Splitting the boundary or introducing a saga fixes the conflicts at the source far better than tuning retries.
  • Snapshot invalidation needs an operational plan for long streams. The “snapshots fall back to replay” safety net is wonderful until a format change makes a million long-stream aggregates each replay from zero at once. For genuinely long streams, plan to rebuild snapshots rather than rely on lazy fallback.

Should you use this?

Situation Recommendation
Any event-sourced aggregate with concurrent writers Strong fit (concurrency) — it’s on; just choose retry vs surface
Long, hot streams whose replay cost you’ve measured Strong fit (snapshots)
Short streams (orders, most aggregates) Skip snapshots — premature; concurrency still applies
You haven’t measured a replay problem Skip snapshots — measure first
High-contention section that must not conflict Consider the built-in IAggregateWriteLock (pessimistic per-aggregate write lock) for that narrow spot
Conflict-free, independent data Concurrency control is moot; this isn’t an aggregate

Next steps

The write side is now durable, concurrent-safe, and fast to load. But every query so far has gone through the repository’s replay, which answers only “give me this aggregate.” Real applications need “give me all accounts over $1000,” “show the activity feed,” “the dashboard” — queries the event store cannot serve. The next article builds the read side: projections that subscribe to the event stream and maintain query-optimized read models.

➡️ 007 — Projections