Articles · Event Sourcing ·024

Schema Evolution: Upcasters, Stable Names, Migrations

Runnable sample on GitHub

Sample: samples/024-schema-evolution · Subsystem: 2.3 · Prerequisites: 005 — Event Sourcing Basics

Overview

In an event-sourced system the events are the database, and unlike rows you can never UPDATE them — history is immutable. So the moment your code needs an event to look different (a renamed field, a new required value, a split type), you have a problem: millions of old events on disk still have the old shape. Relay solves it with two mechanisms:

  • Stable type names — [EventType("name")] decouples the persisted name from the CLR type, so you can rename and refactor classes without orphaning history.
  • Upcasters — IEventUpcaster transforms an old payload into the current shape on read, before deserialization, so the rest of the system only ever sees the latest version.

This sample demonstrates both with the pure serializer — no database needed.

Why this exists

The naive approach — “just deserialize the JSON into the current class” — breaks the first time a field is renamed: old events have LegacyAmount, the class has Amount, and they silently deserialize to 0. The next failure mode is renaming the class: if the persisted type name is the CLR FullName, moving OrderPlaced to a new namespace makes every historical OrderPlaced unresolvable. Teams then reach for risky “migrate the event store” batch jobs that rewrite immutable history. Upcasting + stable names make evolution a read-time concern: history stays untouched, and a small, testable function brings each old event up to date as it’s loaded.

When to use this

  • Any event-sourced aggregate whose events will change shape over time (i.e. all of them, eventually).
  • When you rename or move event/aggregate types and need existing streams to keep loading.
  • When you split or merge events (one old event becomes two, or vice versa).

When not to use this

  • Don’t upcast to fix a bug in history. If past events recorded wrong facts, that’s a corrective event (a new event that adjusts state), not an upcaster. Upcasters change shape, not truth.
  • Don’t upcast read models. A projection read model is disposable — rebuild it from history (article 008) instead of upcasting its rows.
  • Don’t use it for cross-service contracts. Integration events on the wire are versioned differently (consumers must tolerate unknown fields); upcasting is for your event store.

Concepts

Stable names. EventTypeRegistry is a bidirectional map between CLR types and persisted strings. [EventType("samples.order-shipped")] sets the name; absent it, the CLR FullName is used (and a rename orphans history). Resolution is registry-only — there is deliberately no Type.GetType fallback, so a malicious or corrupt stored name can never instantiate an arbitrary type. The registry also rejects collisions: two types claiming one name throws at registration, because the alternative (last-write-wins) would silently deserialize history into the wrong type.

The upcaster chain. IEventUpcaster has two methods:

bool CanUpcast(string typeName);                          // does this upcaster apply to this stored name?
(string TypeName, JObject Json) Upcast(string typeName, JObject json);  // rewrite name + payload

On read, DefaultEventSerializer parses the stored JSON into a JObject and runs the chain in a loop: each applicable upcaster rewrites the payload and may change the type name, until none applies; then it resolves the final name to a CLR type and deserializes. Chains compose — v1→v2→v3 is three upcasters, each handling one hop — and the loop is bounded (64 passes) so a misconfigured cycle can’t hang.

Read-time, not migrate-time. History on disk never changes. The transformation happens every time an old event is loaded. That’s a tiny CPU cost per load in exchange for never running a destructive batch migration over immutable data.

Architecture

flowchart LR
    Store[(relay_events<br/>stored JSON + type name)] -->|read| Ser[DefaultEventSerializer]
    Ser -->|parse| J[JObject]
    J --> Loop{any upcaster<br/>CanUpcast typeName?}
    Loop -->|yes| Up[upcaster.Upcast<br/>rewrite fields + name]
    Up --> Loop
    Loop -->|no| Reg[EventTypeRegistry<br/>name → CLR type]
    Reg --> De[deserialize to current event]
    De --> App[aggregate / projection]

Building it step by step

1. Pin a stable name

[EventType("samples.order-shipped")]
public sealed record OrderShipped(Guid OrderId) : DomainEvent { public override Guid AggregateId => OrderId; }

Now OrderShipped can move namespaces or be renamed; the stored name stays samples.order-shipped.

2. Write an upcaster for an old shape

The v1 payload was stored as samples.invoice-raised.v1 with a LegacyAmount field. The current event is InvoiceRaised(Guid InvoiceId, decimal Amount):

public sealed class InvoiceRaisedV1Upcaster : IEventUpcaster
{
    public const string V1TypeName = "samples.invoice-raised.v1";
    public bool CanUpcast(string typeName) => typeName == V1TypeName;

    public (string TypeName, JObject Json) Upcast(string typeName, JObject json)
    {
        json["Amount"] = json["LegacyAmount"];   // rename the field
        json.Remove("LegacyAmount");
        return (typeof(InvoiceRaised).FullName!, json);  // map forward to the current type
    }
}

3. Register and read

var registry = new EventTypeRegistry();
registry.Register(typeof(InvoiceRaised));
var serializer = new DefaultEventSerializer(registry, new IEventUpcaster[] { new InvoiceRaisedV1Upcaster() });

var restored = serializer.DeserializeEvent(storedV1Event);  // → InvoiceRaised { Amount = 100 }

In production you don’t construct these by hand — AddRelay(assembly) scans your event assemblies into the registry, and you register upcasters with DI (services.AddSingleton<IEventUpcaster, ...>()); the event store wires them into the read path automatically.

Complete source code

File Shows
Events.cs [EventType] stable name + the v1 upcaster
UpcastingTests.cs v1 payload upcast on read; current event round-trips
StableNameTests.cs name round-trip; unknown name; collision rejection

Running the example

dotnet test samples/024-schema-evolution/SchemaEvolution.Sample.Tests

No database — the serializer, registry and upcasters are pure.

Testing

Upcasting is the rare infrastructure concern that is trivial to unit-test: construct an EventData with the old type name and old JSON, deserialize, assert the current shape. Do this for every upcaster you write, with a frozen copy of the real historical payload (copy it out of the database). The test is your guarantee that a five-year-old event still loads — and it runs in microseconds with no infrastructure. The framework additionally proves the path end-to-end through Postgres; the unit test is where you cover each transformation.

Production considerations

  • Add [EventType] from day one. Retrofitting stable names after events are persisted under CLR FullName means writing an upcaster just to fix the name. Cheap up front, annoying later.
  • One upcaster per version hop. Keep v1→v2 and v2→v3 separate; the chain composes them. A single “v1→v3” upcaster duplicates logic and rots.
  • Freeze test fixtures from real data. An upcaster tested against a hand-written payload can still miss a quirk of the actual historical JSON. Copy a real old row into the test.
  • Never delete an old upcaster while any event of that version remains in the store — which, for an event store, is forever. Upcasters accumulate; that’s expected and fine (they’re tiny).
  • Collisions fail loud at startup. If two events claim one [EventType] name, registration throws. That’s the system protecting history — rename one.

Common mistakes

  • Renaming a field without an upcaster — old events silently deserialize the missing field to its default (0, null). No error, just wrong data. The most insidious schema bug there is.
  • Using CLR FullName as the contract — then a refactor (move/rename) orphans history. Always [EventType].
  • Upcasting to rewrite facts — that corrupts the audit log. Use a corrective event.
  • Mutating the shared JObject assumption — the serializer passes a clone; return the (possibly same) object you mutated. Don’t hold references across calls.

Tradeoffs

  • Read-time cost vs. migration risk. Upcasting adds a little CPU on every load of an old event, forever, in exchange for never running a destructive migration. For event stores this is almost always the right trade — immutable history is the whole point.
  • Accumulating upcasters vs. a “big bang” rewrite. Some teams periodically snapshot-and-truncate (fold history into a snapshot, archive the raw events) to retire old upcasters. Powerful but operationally heavy; most never need it.
  • Stable names vs. convenience. [EventType] is one extra attribute per event and a name you must keep forever. Trivial cost; it’s the foundation everything else rests on.

Alternatives

  • Weak schema / tolerant reader (just ignore unknown fields, default missing ones) — works for additive changes but silently corrupts on renames/semantic changes. Upcasters make the change explicit and tested.
  • Versioned event types (OrderPlacedV2 as a distinct class consumers switch on) — viable but pushes version-handling into every reader; upcasting centralizes it at the boundary.
  • Migrating the event store (rewrite old rows) — violates immutability, is risky and irreversible, and is exactly what upcasting exists to avoid.

Lessons from production systems

  • The teams that didn’t add stable names early all wrote the same first upcaster: one that does nothing but fix a type name after a refactor. Add [EventType] now.
  • The scariest schema bug is the silent one — a renamed field deserializing to 0 in a financial event. A per-upcaster unit test against real historical JSON is the cheapest insurance you’ll ever buy.
  • Upcasters are write-once and essentially never change, so they accumulate harmlessly. Resist the urge to “clean them up” — an old upcaster is load-bearing for old events, which live forever.

Should you use this?

If you… Then…
are event sourcing yes — [EventType] everything, upcast on every shape change
only changed an event additively a tolerant reader may suffice, but an upcaster is clearer
recorded a wrong fact emit a corrective event, don’t upcast
want to retire old versions snapshot-and-truncate (advanced); most don’t need it

Next steps