Skip to content

Topic Catalog

Topic directory and register — instantiates Topic-Brokered Event Distribution

A browsable, governed directory of the topics that exist — their meaning, owner, and schema — so teams discover and reuse the right topic instead of inventing a duplicate.

A Topic Catalog is the directory of what topics exist and what each one means. In a system built on a shared topic namespace, the scarce resource is not the ability to create a topic — it is knowing which topic to use. Without a catalog, a team that needs "the customer changed their address" event either cannot find the topic that already carries it or cannot tell whether the three similarly named topics are the same thing, and so it mints a fourth. Its defining move is to make the namespace legible and governed: every topic has a discoverable entry with an owner, a description, a schema pointer, and a lifecycle status, and new topics must earn a well-formed, owned entry before they exist. It is the control plane for the set of topics, not for the events flowing through any of them.

Example

A large enterprise runs several hundred topics across dozens of teams. A new analytics team needs address-change events. Rather than creating cust-addr-update-v3 — the fate of every previous team that couldn't find what already existed — they search the catalog, land on the canonical customer.address_changed entry, and read its owner (the Accounts platform team), its description, its linked schema, and its status (active, not deprecated). They subscribe to the real topic in an afternoon. When that same team later wants to publish a genuinely new event, the catalog requires them to name it to convention, name an owner, and link a schema before the topic can be registered — so the entry they create is one the next team will actually be able to find and trust.

How it works

  • Searchable register — every topic carries structured metadata: owner, human description, domain, schema pointer, and lifecycle status.
  • Naming-convention enforcement — registration checks names against a shared scheme, so the namespace stays consistent[1] instead of accreting ad-hoc labels.
  • Ownership and lifecycle — no topic exists without a named owner, and status (active / deprecated / retired) is recorded so consumers can avoid building on something being sunset.
  • Governance gate at creation — topics come into being through the catalog, making it the point where naming, ownership, and duplication are adjudicated.

Tuning parameters

  • Naming-convention strictness — rigid schemes keep the namespace tidy but frustrate teams with edge cases; loose ones are frictionless but drift toward chaos.
  • Registration gate — self-serve on convention-check versus board approval, trading speed of topic creation against curation.
  • Required metadata — how much a topic must declare (owner alone, or owner plus schema plus SLA); richer entries are more useful but raise the bar to publish.
  • Deprecation visibility — how loudly retirement status is surfaced to existing subscribers, shaping how safely a topic can be sunset.
  • Authority of the catalog — advisory directory versus the sole path to topic creation; the latter keeps it authoritative but demands broker integration.

When it helps, and when it misleads

Its strength is discovery and reuse: teams find and build on canonical topics, duplicates and owner-less "mystery topics" stop proliferating, and the namespace remains something a newcomer can reason about. Its failure mode is drift — a catalog that falls out of sync with reality is worse than none, because it confidently points readers at topics that are dead or omits ones that are live, and trust in it collapses the first time it lies. A related misuse is letting the governance gate harden into bureaucracy so heavy that teams route around it, spawning exactly the shadow topics the catalog exists to prevent. The discipline is to make the catalog authoritative by construction — wiring registration into topic creation so a topic literally cannot exist off-catalog — while keeping the gate light enough that no one wants to evade it.

How it implements the components

  • topic_namespace — it is the organized, discoverable directory of the namespace: the map from a topic's name to its meaning, owner, and schema.
  • topic_governance_board — it is the register through which topic governance is enacted and recorded — naming standards, ownership assignment, and deprecation status all live and are enforced here.

It documents and governs which topics exist, but does not route events over them (broker_routing_boundary — Topic Exchange or Event Bus), define or version their payload schemas (schema_compatibility_matrix — that's Schema Registry), or control who may publish and subscribe (authorization_and_visibility_policy — that's Access-Controlled Topic).

Notes

A catalog governs which topics exist; a schema registry governs what a given topic's events look like. They are complementary and often linked — the catalog's entry points at the registry's schema — but conflating them produces either a directory that cannot validate messages or a schema store no one can discover. Keep the catalog the answer to "which topic?" and let the registry answer "which shape?"

References

[1] The idea of a ubiquitous language — a single, shared vocabulary aligned to the business domain, used consistently in names and models — comes from domain-driven design. A topic namespace is where that discipline either holds or breaks: consistent, domain-aligned topic names are what let a catalog function as a map rather than a phone book of near-synonyms. ↩