Articles · Event Sourcing ·035

Crypto-Shredding for the Right to Erasure

Runnable sample on GitHub

Sample: samples/035-crypto-shredding-right-to-erasure · Subsystem: Security · Prerequisites: 005 — Event Sourcing Basics

Overview

An event-sourced system has a property its operators love and its compliance officers fear: the log is immutable and never deletes. Article 005 made the case — events are the source of truth, they are appended and replayed, and “you cannot edit history” is a feature. Then a data subject invokes GDPR Article 17, the right to erasure, and asks you to delete everything you hold about them. Their name, email, and address are baked into CustomerRegistered, AddressChanged, and a hundred other events that your projections, snapshots, and backups all depend on. Rewriting the log to scrub them is exactly the mutation event sourcing was designed to forbid.

Crypto-shredding resolves the contradiction. Instead of storing PII as plaintext in the events, you encrypt it under a key bound to the data subject. The events still hold the (encrypted) bytes; the ability to read them depends entirely on the key. To “erase” a subject you destroy their key. The immutable log is untouched — every event is byte-for-byte where it was — but the PII inside it is now ciphertext with no key, which is to say: noise. Erasure becomes a single-row delete in a key store, not a rewrite of an append-only event stream.

Relay ships the two primitives this needs in Nuvora.Nexus.Relay.Core.Security:

  • ICryptoShredder (CryptoShredder) — AES-256-GCM encrypt/decrypt of a payload under a subject’s key.
  • ICryptoKeyStore (InMemoryCryptoKeyStore) — the per-subject key store whose ShredAsync is the erasure operation.

Both are pure: no database, no broker, no wall-clock. This sample uses them directly.

Why this exists

The naïve answers to “delete this person’s data from an append-only log” are all wrong:

  • Rewrite the events. This breaks the one invariant the whole pattern rests on. Snapshots, the global position cursor, downstream consumers, and every projection that already folded those events are now inconsistent with a history that silently changed underneath them.
  • Hard-delete the events. Same problem, plus you punch holes in a contiguous, version-numbered stream (article 005: versions are 0, 1, 2… and replay assumes no gaps).
  • Keep a “deleted” flag. The PII is still there in plaintext — you have not erased anything, you have hidden it from one query path. That is not what Article 17 means, and an auditor or a leaked backup will say so.

Crypto-shredding gives you a legally meaningful deletion that is also operationally cheap: the data is rendered unrecoverable by destroying ~32 bytes of key, the event log keeps its integrity, and you can demonstrate to a regulator exactly which key was destroyed and when.

When to use this

  • Event-sourced or otherwise append-only stores that must honour the right to erasure (or any “make this unrecoverable” requirement) without mutating history.
  • PII embedded in immutable records — events, audit logs, write-once archives — where selective in-place deletion is impossible or forbidden.
  • Per-subject erasure — you need to erase this person without touching anyone else’s data. A per-subject key gives you exactly that blast radius.
  • Backups and replicas you cannot reach. Once the key is gone, every copy of the ciphertext — in last night’s backup, in a read replica, in a downstream consumer’s store — is equally unreadable. You do not have to chase the data across systems; you only have to destroy one key.

When not to use this

  • Plain CRUD where you can just DELETE. If the PII lives in a mutable row you fully control, deleting the row is simpler and more honest than encrypting it. Crypto-shredding earns its keep only where in-place deletion is impossible.
  • Data you are legally required to retain. Erasure and retention are in tension (financial records, tax law). Crypto-shredding does not resolve that conflict — it just gives you a clean lever once you have decided erasure is permitted.
  • When you have not first stopped PII leaking elsewhere. If the same name also sits in plaintext in a projection, a log line, an email gateway, or event metadata, shredding the event-body key erases nothing meaningful. The technique is only as strong as your discipline about where plaintext lives.
  • As a substitute for encryption-at-rest or access control. This is erasure, not confidentiality layering. You still want at-rest encryption and authorization (article 010) around the data while it is live.

Concepts

Per-subject key. Each data subject gets their own random AES-256 key, created lazily on first use (GetOrCreateKeyAsync). The key — not the data — is the unit of erasure. Because keys are per subject, destroying one affects exactly one person’s data and no one else’s.

AES-GCM envelope. CryptoShredder encrypts with AES-256-GCM, an authenticated cipher. Each call draws a fresh random 12-byte nonce, and GCM produces a 16-byte authentication tag. Authentication means tampering is detected on decrypt rather than silently returning garbage. Because the nonce is random per call, encrypting the same plaintext twice yields different ciphertext.

Packaging (nonce | tag | ciphertext). Decryption needs the nonce and tag, so they travel with the ciphertext. CryptoShredder concatenates nonce (12) | tag (16) | ciphertext (n) and Base64-encodes the whole thing into the string you store in the event. On decrypt it slices the three pieces back out. You store one opaque Base64 blob and never have to manage nonces yourself.

Erase = destroy the key. ICryptoKeyStore.ShredAsync(subjectId) removes the subject’s key. After that, FindKeyAsync returns null, so CryptoShredder.TryDecryptAsync returns null — the contract for “erased / unreadable.” Crucially, nothing in the event log changed; only the key store did.

Recoverable vs. unrecoverable, by contract. While the key exists, TryDecryptAsync returns the original plaintext. Once shredded, it returns null. There is no third state — no “partially erased,” no way to bring the data back. That binary, key-gated outcome is what makes the erasure claim defensible.

Architecture

flowchart TD
    subgraph Write["Encrypt PII on the write path"]
        PII[Plaintext PII<br/>'Ada, ada@example.com'] --> Enc[CryptoShredder.EncryptAsync]
        KS1[(ICryptoKeyStore)] -->|GetOrCreateKey subject-X| Enc
        Enc -->|nonce . tag . ciphertext, Base64| Log[(Append-only event log<br/>immutable)]
    end

    subgraph Read["Decrypt on replay / projection"]
        Log --> Dec[CryptoShredder.TryDecryptAsync]
        KS2[(ICryptoKeyStore)] -->|FindKey subject-X| Dec
        Dec -->|key present| Plain[Recovered plaintext]
        Dec -->|key shredded → null| Gone[Unrecoverable noise]
    end

    Erase[Right-to-erasure request<br/>subject-X] -->|ShredAsync subject-X| KS2
    Erase -. log is never rewritten .-> Log

The event log is written once and never edited. Erasure flows entirely through the key store: drop the key, and every event encrypted under it — past, replicated, or backed up — becomes unreadable at once.

Building it step by step

The full sample is in samples/035-crypto-shredding-right-to-erasure.

1. Wire the shredder over a key store

var keys = new InMemoryCryptoKeyStore();   // per-subject AES-256 keys (swap for a durable store in prod)
var shredder = new CryptoShredder(keys);   // ICryptoShredder over that key store

2. Encrypt PII on the write path

// Store this opaque Base64 string in the event instead of the raw PII.
string cipher = await shredder.EncryptAsync("subject-X", "Ada Lovelace, ada@example.com");

3. Decrypt on replay / projection

string? recovered = await shredder.TryDecryptAsync("subject-X", cipher);
// recovered == "Ada Lovelace, ada@example.com"  (while subject-X's key exists)

4. Erase the subject — destroy the key

await keys.ShredAsync("subject-X");                       // the right-to-erasure operation
var gone = await shredder.TryDecryptAsync("subject-X", cipher);   // → null, unrecoverable

The event holding cipher is never touched. Only the key store changed.

Complete source code

File Shows DB?
CryptoShreddingTests.cs protect ≠ plaintext; round-trip; erase → unrecoverable; per-subject isolation No
Privacy.CryptoShredding.Tests.csproj references Nuvora.Nexus.Relay.Core only No

Running the example

dotnet test samples/035-crypto-shredding-right-to-erasure/Privacy.CryptoShredding.Tests

This needs only the .NET 10 SDK — no database, pure crypto + an in-memory key store. That is deliberate: crypto-shredding’s guarantee lives in the cipher and the key lifecycle, not in any storage engine, so the honest demonstration is a fast, deterministic unit test against the real primitives.

Testing

The sample’s CryptoShreddingTests.cs proves the four properties that make the technique trustworthy.

Protecting PII produces ciphertext, and it round-trips while the key lives:

var keys = new InMemoryCryptoKeyStore();
var shredder = new CryptoShredder(keys);

var cipher = await shredder.EncryptAsync("subject-X", "Ada Lovelace, ada@example.com");
cipher.Should().NotBe("Ada Lovelace, ada@example.com");                 // it's encrypted
cipher.Should().NotContain("Ada Lovelace");                             // no plaintext fragment survives

(await shredder.TryDecryptAsync("subject-X", cipher))
    .Should().Be("Ada Lovelace, ada@example.com");                      // recovers while the key exists

Erasing the key makes the data unrecoverable — without rewriting any event:

await keys.ShredAsync("subject-X");

(await shredder.TryDecryptAsync("subject-X", cipher)).Should().BeNull(); // null == erased
(await keys.FindKeyAsync("subject-X")).Should().BeNull();                // the key itself is gone

Erasure is per subject — one subject’s deletion does not touch another’s:

var cipherY = await shredder.EncryptAsync("subject-Y", "y-pii");
await keys.ShredAsync("subject-X");

(await shredder.TryDecryptAsync("subject-X", cipherX)).Should().BeNull();      // X erased
(await shredder.TryDecryptAsync("subject-Y", cipherY)).Should().Be("y-pii");   // Y unaffected

These are pure tests against the real CryptoShredder and InMemoryCryptoKeyStore — no mocks, because the property under test (decrypt returns null exactly when the key is gone) is a property of the real crypto and key lifecycle. Mocking the key store would test the mock.

Production considerations

  • Use a durable, shared key store. InMemoryCryptoKeyStore is for single-node use and tests; restart it and every key is gone — that is accidental mass erasure. Production needs the keys persisted in a durable, replicated store (a database table, a KMS/HSM, a secrets manager), shared across every node that must encrypt or decrypt. Multi-node erasure in particular requires a shared store, or one node will happily keep decrypting with a key another node “shredded.”
  • Back up the keys — but understand the tension. Keys are now your single point of failure: lose them and you have erased everyone. So you back them up. But a key backup is, by definition, a way to undo erasure — if a shredded key survives in a backup, the subject is not truly erased once that backup is restored. Reconcile this explicitly: either expire/rotate backups so shredded keys age out within your stated retention window, or actively purge shredded keys from backups as part of the erasure workflow.
  • Know what metadata still leaks. Crypto-shredding erases the encrypted payload, not the surrounding facts. The event still records that subject-X existed, when they did things, the event types, the stream id, the timestamps, and the global position. If the subjectId, an email, or a name sits in event metadata, headers, or projection keys in plaintext, shredding the body key erases nothing about them. Treat the subject identifier as a pseudonymous token, keep PII out of metadata, and audit your projections (article 007) for plaintext copies.
  • Key rotation. Re-encrypting a subject’s data under a fresh key periodically limits the blast radius of a single key compromise. Note rotation interacts with erasure: if you rotate, “erase” must destroy every key version that ever protected the subject, not just the current one. Keep the key→subject mapping authoritative so erasure can find them all.
  • Encrypt at rest and authorize anyway. Shredding is about erasure, not confidentiality while data is live. Keep at-rest encryption on the stores and access control (article 010) around the read paths; crypto-shredding is the deletion story layered on top, not a replacement for either.

Common mistakes

  • Leaving plaintext PII somewhere else. The classic failure: you dutifully encrypt the event body, then a projection materialises customer_name in plaintext, a log line prints the email, or the outbound email gateway keeps a copy. Shredding the event key leaves all of those intact, and you have not erased the subject. Inventory every place PII lands and route it all through the shredder (or keep it out entirely).
  • Putting PII in metadata. Event headers, the subjectId itself, stream names, and projection primary keys are not encrypted by the body shredder. A plaintext email used as a stream key survives erasure. Use opaque, non-identifying tokens for keys and metadata.
  • Losing keys = accidental mass erasure. Treating the key store as ephemeral (in-memory, un-backed-up, not replicated) means a restart or a node loss silently and permanently erases real users. Durability of the key store is a correctness requirement, not an optimisation.
  • Over-broad keys. One key for many subjects (or “all EU users”) means you cannot erase one person without erasing the group. The blast radius of erasure equals the scope of the key — keep it per subject.
  • Forgetting old key versions when rotating. If you rotate keys but erasure only destroys the latest, data encrypted under a prior version stays readable. Erasure must sweep every version.

Tradeoffs

Benefits. Right-to-erasure over an append-only / event-sourced store without mutating history (article 005’s invariant stays intact); erasure that propagates automatically to backups and replicas (no key, no read, anywhere); a clean, auditable deletion event (which key, when); and a tiny erasure operation (destroy ~32 bytes) instead of a log rewrite.

Costs. A new critical dependency — the key store — whose loss is catastrophic and whose backups are in tension with erasure; the discipline to keep all PII inside the encrypted envelope and out of metadata; per-record crypto overhead on read and write; and residual metadata (timestamps, event types, existence) that crypto-shredding does not erase. It is a deletion mechanism, not a privacy panacea.

Alternatives

  • Tokenization / pseudonymization. Replace PII with a token and keep the token→PII mapping in a separate vault; erase by deleting the mapping. Similar key-lifecycle story, often paired with crypto-shredding (tokenize identifiers, encrypt free-text payloads).
  • Separate PII store, events hold only a reference. Keep no PII in events at all — store a foreign key to a normal, mutable table and DELETE the row to erase. Simple and clean if you can keep PII out of the event body entirely; the moment a free-text field contains PII, you are back to crypto-shredding.
  • Event rewriting / log rewriting. Actually scrub or rewrite the offending events (a “stream upgrade”). Possible, but it violates immutability, invalidates snapshots and downstream consumers, and is operationally fraught — see article 024 (schema evolution) for why editing history is the path you work hard to avoid. Crypto-shredding exists precisely so you do not have to.

Lessons from production systems

  • The teams that get burned are the ones who shredded the event key and declared victory while the same name sat in plaintext in three projections, a search index, and last week’s CSV export. Erasure is a data-flow problem; the cipher is the easy 20%.
  • The key store quietly becomes the most important database you run. The first time it is “just an in-memory cache” and a deploy restarts the fleet, you have erased every user at once. Make it durable, replicated, and backed up before you ship.
  • Backups and erasure will fight, and the fight is a policy decision, not a code one. Decide up front how long a shredded key may survive in a backup, write it down, and enforce it — auditors ask, and “we never thought about it” is the wrong answer.

Should you use this?

If you… Then…
run an event-sourced / append-only store with PII and owe a right-to-erasure yes — crypto-shred per subject
have PII in immutable records you cannot selectively delete yes
can simply DELETE a mutable row to erase prefer that — simpler and more honest
are legally required to retain the data crypto-shredding can’t resolve the conflict; decide policy first
haven’t yet stopped PII leaking to projections/logs/metadata fix that first — shredding the body key won’t help

Next steps

  • 005 — Event Sourcing Basics: the immutable, append-only log this technique protects — and why “you cannot edit history” makes crypto-shredding necessary.
  • 007 — Projections: the most likely place plaintext PII escapes the envelope; audit your read models for copies the shredder cannot reach.
  • 010 — Authorization: confidentiality and access control around live data — the layer crypto-shredding sits on top of, not a replacement for.
  • 024 — Schema Evolution: why rewriting history (the alternative to crypto-shredding) is the path you work to avoid.