Skip to main content

Consistency boundaries

A consistency boundary is the set of aggregates one write commits atomically: all of them land, or none do. Most event stores draw it at one aggregate. Celeriant draws it at one shard, which holds many aggregates, and the routing rule is how you decide which ones.

Beyond the single aggregate​

Classic event sourcing draws the boundary at one aggregate: each write touches one stream, and any invariant spanning two becomes a saga. Celeriant lets one write request carry conditional writes to several aggregates and commit them together.

The textbook case is a transfer: debit one account, credit another, both or neither. You send one request carrying both, each guarded on its own expected version. The shard validates every aggregate in the request (existence, version, idempotency, schema) before it appends anything. One failure rejects the whole request and neither stream changes. Pass, and every batch is queued in one step and acknowledged together once fsync and replication finish. No saga, no compensating action, no half-applied transfer to clean up. The multi-aggregate writes guide has the code.

The pattern goes by dynamic consistency boundaries (DCB). For the idea behind it, Sara Pellegrini and Milan Savic's talk is worth the time.

The boundary is a shard​

One atomic write spans one shard. That is the line. Cross-shard atomic writes would bring back the distributed transaction this database exists to avoid, so the server rejects them at routing, before any shard sees the request: ShardRoutingMultipleShards, error 9001.

You decide which aggregates share a shard. Placement is routing_id % num_shards, where routing_id is the part of the key the routing rule names (--routing-rule, fixed on first start). Plain % on an id you control, not a hash, so you can put co-committed aggregates together on purpose. With --reserve-coordinator-shard the formula becomes routing_id % (num_shards - 1) + 1; do the arithmetic against the formula your cluster actually runs.

Pick the rule that matches the invariants you enforce together:

  • org_id: every aggregate in one org lands on one shard. Transfers between accounts in the same org just work. Cost: that org's writes serialise on one core. Right when invariants are per tenant and a tenant's write rate fits one core.
  • aggregate_type_id: every aggregate of a type lands on one shard. Right when atomic writes span aggregates of one type.
  • aggregate_id (default): even spread by aggregate id. Right when most writes touch one aggregate and you want the cores. Multi-aggregate writes still work, but you have to engineer the id space so aggregates that commit together land together: aggregate_id = 1000 and aggregate_id = 1004 on a 4-shard cluster both go to shard 0. Pick ids at random and multi-aggregate writes fail with 9001.

Choosing the rule is the architectural decision. Choosing the ids is the discipline that follows. If an invariant spans shards however you route, this is the wrong tool for that write; coordinate it outside the store.