Skip to main content

Schema formats

The server enforces a registered schema on every write to its event type. This page is the exact contract: the key, the three formats, the string each one expects, and what a payload has to pass. For the concept see Schema validation; for the recipe, Registering and evolving schemas.

The key

A schema belongs to a SchemaKey:

org_id / aggregate_type_id / event_type_major / event_type_minor

There is no separate event-type name. An event type is a (major, minor) pair under an aggregate type, so each version gets its own schema and old and new shapes validate side by side while you migrate.

Formats

SchemaType is a single byte on the wire:

SchemaTypeByteServer compiles withSchema string
Json0jsonschemathe JSON Schema document
Avro1apache_avrothe Avro schema JSON
Protobuf2prost_reflectbase64(FileDescriptorSet):MessageName

Any other byte fails with RegisterSchemaUnsupportedType (2024).

Examples of the string:

  • Json: {"type":"object","properties":{"amount":{"type":"integer"}},"required":["amount"]}
  • Avro: {"type":"record","name":"Order","fields":[{"name":"amount","type":"long"}]}
  • Protobuf: not .proto source. Standard base64 of a serialized FileDescriptorSet (what protoc --descriptor_set_out writes), a colon, then the fully qualified message name: CqYB...:shop.Order. The CLI builds this string for you from --proto-descriptor order.pb --message-name shop.Order.

Registration

The definition is capped at 16 KiB (--max-schema-size-bytes, default 16384). Exactly at the limit is accepted; one byte over fails.

The server compiles the schema before storing it. A string that does not parse, a bad base64 blob, a message name missing from the descriptor set, or an oversized schema all fail with RegisterSchemaInvalid (2021).

Registration always lands on shard 0, which appends a schema record to its WAL, then fans the same registration out to every other shard. Each shard keeps its own copy in its own log, so a write validates locally with no cross-shard hop. If any shard fails, the client gets RegisterSchemaCoordinationFailed (2029). The fan-out is not atomic: shard 0 may already hold the schema, and a plain retry then returns RegisterSchemaAlreadyExists (2020).

Schemas are additive. There is no update, delete, or fetch request. Registering a key that already has a schema fails with RegisterSchemaAlreadyExists. To change a shape, register the next (major, minor).

Registration is a leader operation. On a follower it fails with RegisterSchemaCannotAcceptWrites (2027).

What a write has to pass

For each event in a write, the shard looks up the schema for (org, aggregate type, major, minor):

FormatPayload passes when
Jsonthe bytes parse as JSON and the value validates against the schema
Avrothe bytes decode as a raw Avro datum (binary, no container-file header) with this schema, and the decoded value matches it
Protobufthe bytes decode as the named message. Unknown fields pass, missing fields pass, empty bytes pass

One failing event rejects the whole write with WriteSchemaValidationFailed (2022), naming that event's major, minor and client seq. Nothing is appended.

Validation does not happen when:

  • No schema is registered for the key. The event is written as is. Validation is opt-in per event type.
  • The event is encrypted. An event carrying an IV is skipped; the server cannot read the plaintext. Per event type, validate or encrypt, not both.

Protobuf validation is structural. It proves the bytes decode as the message, not that any field is set.

WriteSchemaCompilationFailed (2023) means the stored schema failed to compile when the shard loaded it back from its log. Writes to that event type fail with it. See the error codes.