Skip to main content

The audit chain

Every shard's WAL is a BLAKE3 hash chain. Change, insert or drop an entry and the recomputed tip stops matching. Tampering is not prevented, it is made detectable.

What is chained​

The unit is the metablock: one per write batch, delete tombstone, trim tombstone or schema registration. Each metablock stores previous_tip_hash, and the new tip is blake3(previous_tip_hash || metablock bytes).

The first entry chains from GENESIS_HASH, 32 zero bytes. The tip carries across segment rotation, so the chain runs unbroken over a shard's whole history. Shards chain independently.

Three things sit outside the hash: the block CRC, and two node-local fields (datablock_position, previous_aggregate_metablock_pos). That is what lets leader and follower produce the same tip from different on-disk layouts.

Payload coverage depends on size:

  • Small batches are stored inline in the metablock, so the payload bytes are hashed directly.
  • Larger batches live in a separate datablock. The metablock carries that datablock's CRC32C, and the CRC is what gets hashed. Accidental corruption is caught. A deliberate edit that preserves the CRC32C is not, because CRC32C is not a cryptographic hash.

What the server does with it​

Divergence detection, and nothing else:

  • A follower rejects a replication batch whose previous_tip_hash does not match its own tip (TipHashMismatch), then recovers through S3 catch-up, which walks back to the common ancestor and truncates the fork.
  • The leader checks a batch's internal chain continuity before shipping it to S3.

Boot does not re-walk the chain. A node loads the tip persisted in its segment header and trusts it. The tip is not exposed through the client API.

Compaction ends verifiability for a segment​

Compaction rewrites a sealed segment without the batches of deleted or trimmed aggregates. It keeps the segment's original tip in the header, but the surviving metablocks no longer chain to it. After compaction, recomputation proves nothing for that segment. See Retention and deletion.

Verifying​

No built-in verify command exists. celeriant-wal-inspect <log_file> header prints a segment's write and read tip hashes, and range <start> <end> prints each metablock's previous_tip_hash. Comparing tips across the two nodes is what the chaos harness does after every run to catch forks.

A full recompute needs the raw WAL files and a script, compared against a reference an attacker could not also rewrite: the other node, a backup, or a tip hash recorded elsewhere. The chain shows that history changed, not who changed it. See Verifying the audit chain.