Skip to content

Schema Registry

Schema register — instantiates Topic-Brokered Event Distribution

A managed register of event schemas and their versions that decides whether a new message format is compatible before producers and consumers ever exchange it.

A Schema Registry is the authority that holds the shape of the events flowing through the broker: for each topic, the schema every message must conform to, every version that schema has had, and the rules governing how it may change. Its distinguishing job is compatibility enforcement at the moment of change — when a producer proposes a new schema version, the registry checks it against the compatibility policy and rejects the change if it would break existing consumers, rather than discovering the break at runtime when a consumer chokes on an unexpected field. In a system where producers and consumers are decoupled and never speak directly, the registry is the only place their agreement about message structure is written down and defended.

Example

A payments platform emits PaymentAuthorized events consumed by ledgering, fraud, and notification services that the payments team has never met. The team needs to add an installment_plan_id field. Without a registry this is a gamble — some old consumer might reject the changed message. With one, the team submits the new schema; the registry checks it under a backward-compatible policy, confirms the field is optional with a default, and admits it as version 4. Every existing consumer keeps deserializing without change; new consumers can opt into the field.

Later someone proposes removing the currency field. The registry evaluates it against the same policy, sees that consumers still read currency, and rejects the version before it can be published. The break that would have surfaced days later as silent misposted ledger entries is instead a failed check at commit time, with a clear reason attached.

How it works

  • One record per topic's contract. For each subject the registry stores the canonical schema and its full version history — the single written statement of what a message on that topic looks like.
  • Check compatibility on registration. A proposed schema is evaluated against the prior version(s) under the configured mode (backward, forward, full, or none) before it is allowed to become active.
  • Admit or refuse. Compatible changes are versioned and published; incompatible ones are refused with the offending difference named, so the break never reaches the wire.
  • Serve the schema for (de)serialization. Producers and consumers fetch the exact version they need, so both sides interpret bytes the same way without a shared build.

Tuning parameters

  • Compatibility mode — backward (new consumers read old data), forward (old consumers read new data), full (both), or none. Stricter modes protect more consumers but forbid more kinds of change.
  • Enforcement point — block at publish/registration versus warn and allow. Blocking guarantees safety but can halt a deploy; warning keeps velocity at the cost of runtime risk.
  • Version resolution — whether consumers pin an exact version or float to "latest compatible." Pinning is predictable; floating auto-adopts additive changes but assumes the policy holds.
  • Schema scope — one contract per topic versus multiple allowed subjects per topic. Narrow scope is easy to reason about; broad scope is flexible but weakens the "one topic, one shape" guarantee.
  • Default-and-optional policy — how aggressively new fields must be optional-with-default. Strict rules make evolution safe but verbose.

When it helps, and when it misleads

Its strength is converting schema change from a runtime hazard into a checked, reviewable event: because the registry defends backward and forward compatibility[1] at the point of change, decoupled teams can evolve messages independently without secretly breaking strangers downstream, and every version is discoverable rather than tribal knowledge.

It misleads when the registry is treated as complete protection. Compatibility rules constrain structure, not meaning: repurposing a field to hold something new can pass every compatibility check and still corrupt every consumer, because the bytes still fit. A registry set to "none" or bypassed "just this once" gives false confidence — teams assume messages are safe because a registry exists. And an over-strict full-compatibility mode can ossify a schema no one is allowed to fix. The discipline is to pair structural checks with semantic review of what a field means, keep enforcement actually enforcing, and revisit the compatibility mode as the topic's consumer set changes.

How it implements the components

  • event_message_contract — it holds the canonical, versioned definition of what a message on each topic must look like: the written contract producers and consumers both bind to.
  • schema_compatibility_matrix — it computes and enforces which version transitions are safe, refusing changes that would break consumers under the chosen policy.

It does not route or fan out events (Message Broker, Fan-Out Exchange), select which events a subscriber wants by their contents (Content-Based Subscription Filter), or guarantee a well-formed message was actually delivered (Delivery Acknowledgement).

References

[1] Backward compatibility means a consumer built for an old schema can still read data written under a new one; forward compatibility is the reverse. Enforcing these at registration is standard practice in schema registries (e.g. for Avro-encoded event streams) and is what lets producers and consumers evolve on independent schedules. ↩