Articles · Production Operations ·025

Reference Architecture: A Production Service End-to-End

Runnable sample on GitHub

Sample: samples/025-reference-architecture · Capstone · Prerequisites: this article assumes 001–024

Overview

The previous twenty-four articles each isolated one capability. This one puts them together into the shape Relay is actually for: a service that takes commands, event-sources its aggregates, announces facts to other services through the outbox, builds read models with projections, and answers queries from those read models — every write atomic, no dual writes, observable and resilient by default. The sample is a small “Storefront” order service, and a single end-to-end test drives the whole flow on real PostgreSQL.

Why this exists

Knowing each pattern individually isn’t the same as knowing how they compose. The hard questions are at the seams: How does an integration event commit atomically with the aggregate’s events? How does the read side stay consistent with the write side without distributed transactions? Where do tenancy, authorization, telemetry and retries plug in? This article answers them by wiring one coherent service and showing the data flow from Execute to query result.

The shape

flowchart TD
    subgraph Write side
        Cmd[Command] --> Pipe[Pipeline: authz → tenancy → validation → logging → transaction]
        Pipe --> Handler[Handler]
        Handler -->|raise events| Agg[Order aggregate]
        Handler -->|publish| IB[IIntegrationEventBus]
        Agg -->|append| ES[(relay_events)]
        IB -->|stage| OB[(relay_outbox_messages)]
        ES -. one transaction .- OB
    end
    OB -->|OutboxProcessor| Broker[(message broker)]
    ES -->|catch-up| Proj[Projection host]
    Proj --> RM[(order_summaries read model)]
    subgraph Read side
        Q[Query] --> QH[Query handler] --> RM
    end
    Broker -.->|other services / inbox| Elsewhere

The two atomicity guarantees that make this work:

  1. Command-side: the aggregate’s events and the outbox row are written to the same DbContext and committed by the transaction behavior in one database transaction. A rolled-back command stages nothing — no “saved the order but lost the notification” dual-write bug.
  2. Read-side: the projection writes the read-model row and advances its checkpoint in one transaction, so the checkpoint is never ahead of the data. Catch-up is asynchronous and idempotent, so the read side is eventually consistent with the write side — by design.

Walking the flow

Command → write model (atomic)

PlaceOrderCommand creates the aggregate and publishes a notification:

public sealed class PlaceOrderCommandHandler(IIntegrationEventBus events) : ICommandHandler<PlaceOrderCommand, Order>
{
    public async Task<Order> Handle(PlaceOrderCommand command, CancellationToken ct)
    {
        var order = Order.Place(command.OrderId, command.Customer, command.Total);
        await events.Publish(new OrderPlacedNotification { OrderId = command.OrderId }, ct); // → outbox row
        return order;                                                                         // → OrderPlaced event
    }
}

Returning the Order (an AggregateRoot<Guid>) makes the transaction pipeline append its uncommitted events; the Publish stages an outbox row. Both land in the command’s transaction. The test asserts the outbox row exists and is Pending.

Event stream → read model

OrderSummaryProjection folds the stream into a flat row, using FindAsync so a row inserted earlier in a batch is visible to later events in the same batch (see article 007):

public bool Handles(string eventType) => eventType == typeof(OrderPlaced).FullName || eventType == typeof(OrderPaid).FullName;

public async Task ProjectAsync(EventData @event, CancellationToken ct = default)
{
    var orders = _context.Set<OrderSummary>();
    switch (serializer.DeserializeEvent(@event))
    {
        case OrderPlaced e when await orders.FindAsync([e.OrderId], ct) is null:
            orders.Add(new OrderSummary { OrderId = e.OrderId, Customer = e.Customer, Total = e.Total, Status = "Placed" });
            break;
        case OrderPaid e when await orders.FindAsync([e.OrderId], ct) is { } row:
            row.Status = "Paid";
            break;
    }
}

Read model → query

public Task<OrderSummary?> Handle(GetOrderSummaryQuery query, CancellationToken ct)
    => db.OrderSummaries.AsNoTracking().FirstOrDefaultAsync(r => r.OrderId == query.OrderId, ct);

Queries read the read model, never the event stream. (And because the query pipeline constructs the caching behavior for every query, the service registers AddRelayInMemoryCache() — see article 001.)

One DbContext

modelBuilder.ApplyRelayEventStore();            // write model
modelBuilder.ApplyRelaySnapshots();
modelBuilder.ApplyRelayOutbox();                // integration events
modelBuilder.ApplyRelayProjectionCheckpoints(); // read-side progress
modelBuilder.Entity<OrderSummary>(e => { e.ToTable("order_summaries"); e.HasKey(r => r.OrderId); });

Sharing one context is what makes the cross-cutting writes atomic. In production the outbox processor and projection host run as hosted services (AddRelayProjections()); the sample drives the host by hand so the test is deterministic.

Where the other subsystems plug in

Concern How it attaches Article
Authorization [RequireRole]/[AllowAnonymous] on commands/queries; AddRelayAuth() 010
Multi-tenancy [TenantScoped]; AddRelayTenancy() stamps outbox/projections per tenant 020
Reliable delivery OutboxProcessor → broker → consumer; the inbox dedups on the far side 011–014
Long-running workflows a saga reacts to OrderPlacedNotification (e.g. fulfilment) 015–017
Time a scheduled command (e.g. “cancel if unpaid in 24h”) 018–019
Observability every step emits spans + metrics on RelayTelemetry 021
Resiliency / scale retries/breakers; leader-elected processors; partitioned consumers 022–023
Schema evolution [EventType] + upcasters keep old OrderPlaced events loading 024

None of these change the core flow — they compose onto it.

Complete source code

File Role
Order.cs aggregate + domain/integration events
OrderingCommands.cs commands + handlers (event-source + publish)
OrderSummaryProjection.cs read model + projection
OrderQueries.cs query + handler
StorefrontDbContext.cs one context: events + outbox + checkpoints + read model
StorefrontFixture.cs production DI wiring + Testcontainers
EndToEndTests.cs command → outbox → projection → query

Running the example

dotnet test samples/025-reference-architecture/Storefront.Sample.Tests

Requires the .NET 10 SDK and Docker (PostgreSQL via Testcontainers).

Testing

The test asserts the seams, not just the endpoints: the outbox row exists and is Pending (atomic publish), the read model eventually reflects the order (catch-up), and a second command propagates through (Status = "Paid"). The asynchronous read side is polled with a bounded wait — that’s the honest shape of eventual consistency, and testing it this way is how you build confidence that the pieces actually compose, not just that each unit works in isolation.

Production considerations

  • Run the processors as hosted services. AddRelayProjections() and the outbox processor run for the app’s lifetime; under multiple instances, leader-elect or partition them (article 023) so they don’t duplicate work.
  • Embrace eventual consistency on the read side. A just-placed order may not be queryable for a few hundred milliseconds. Design the UX for it (optimistic UI, “processing…”, read-your-writes only where it matters) rather than trying to make projections synchronous.
  • Idempotency everywhere asynchronous. The outbox is at-least-once; the inbox dedups; projections are idempotent (the FindAsync insert guard). Every async hop assumes redelivery.
  • One context, many concerns — keep it boring. Resist splitting the event store and read models across databases unless you have a measured reason; the shared transaction is the simplicity that buys you atomicity.
  • Observe the seams. Alert on outbox pending depth and projection lag (article 021) — they’re the leading indicators that the async side is falling behind.

Common mistakes

  • Querying the event stream instead of a read model — it works for one aggregate and collapses under list/search/reporting load. Project.
  • Expecting read-after-write consistency from an async projection — it’s eventually consistent; building UI that assumes otherwise produces “I placed it but it’s not there” bugs.
  • Publishing integration events outside a command — there’s no transaction to stage onto, and Relay fails loud rather than drop the event. Publish from within the command.
  • Forgetting AddRelayInMemoryCache() — the query pipeline needs an ICachingStrategy even with nothing cached (article 001).

Tradeoffs

  • CQRS + ES vs. CRUD. You get a perfect audit log, temporal queries, and decoupled read models — at the cost of eventual consistency, more moving parts, and schema-evolution discipline. For a domain where history and integration matter, it’s worth it; for a simple CRUD admin screen, it’s overkill (and articles 001–004 are a fine stopping point).
  • One service vs. many. This sample is one bounded context. Real systems are several, integrating via the outbox/broker/inbox. The patterns are identical; the boundaries are a domain-design question.
  • Sync vs. async reads. Asynchronous projections scale and decouple, but add latency and consistency nuance. Synchronous read models (update in the command) are simpler but couple write and read.

Alternatives

  • CRUD + a transactional outbox (no event sourcing) — keep current-state tables, still get reliable integration. A pragmatic middle ground; you lose the event log and temporal queries.
  • A service bus framework (MassTransit, NServiceBus, Wolverine) — mature messaging/saga stacks. Relay differentiates by integrating the event store, projections, tenancy and the mediator into one coherent whole with a single transaction; if you only need messaging, those are excellent and lighter.
  • A managed event-sourcing database (EventStoreDB, Marten) — purpose-built stores. Relay deliberately uses the PostgreSQL you already run, trading some specialization for one-fewer-dependency operations.

Lessons from production systems

  • The dual-write bug (“saved the row, failed to publish the event”) is the one that quietly corrupts cross-service state for weeks. The single-transaction outbox is the whole reason this architecture exists; never publish outside the command.
  • Teams that fought eventual consistency — bolting synchronous reads onto an async pipeline — built fragile, slow systems. The ones that embraced it (and designed the UX around it) scaled cleanly.
  • The architecture is only as good as its observability. Wire telemetry and health from day one; the seams (outbox depth, projection lag) are where production problems first show, and they’re invisible without it.

Should you use this?

If your domain… Then…
has real workflows, audit needs, and cross-service integration yes — this is the target architecture
is mostly CRUD with simple reporting stop earlier (001–011); add the outbox if you integrate
needs strict read-after-write everywhere reconsider async projections, or scope them carefully
is a single team learning the patterns build exactly this sample, then grow it

Next steps

You’ve reached the end of the guided path — you’ve seen every subsystem in isolation and now composed. From here: pick the bounded context you’re actually building, start from the slice in this sample, and add only the subsystems your domain needs. Re-read the relevant article when you reach each seam. The coverage plan is your map back to any topic.