Contract Test Suite¶

Automated conformance test — instantiates Domain–Codomain Delimitation

Renders the declared boundary as executable cases and counterexamples that fail the build whenever an implementation accepts an out-of-domain input or emits an out-of-codomain output.

A Contract Test Suite turns a boundary that others merely declare into one that is continuously verified. It is a body of executable cases — supported inputs that must be accepted, out-of-domain inputs that must be rejected, and assertions that every output stays inside the codomain — run automatically against an implementation so that any drift breaks the build. Its distinguishing move is that it neither defines the boundary (a signature or schema does) nor enforces it at runtime (a validation gate does): it proves the implementation actually honours the declared boundary, catching the gap between "what we said the domain and codomain were" and "what the code really does." The counterexamples are as load-bearing as the positive cases — a suite that only checks happy-path inputs never demonstrates the boundary holds.

Example¶

An orders service calls a payments service, relying on payments to accept only well-formed charge requests and to return a status that is always one of authorized, declined, or pending. A consumer-driven contract captures that expectation, and the Contract Test Suite runs it against the payments provider on every build: it sends supported requests and asserts the responses; it sends a request with a negative amount and asserts a rejection, not a 500; it asserts the returned status is never some fourth value like unknown. When a payments refactor starts returning status: "processing" — a new codomain member the orders side was never told about — the contract test goes red in the provider's own pipeline, before release, instead of surfacing as a mysterious downstream failure weeks later.^[1] The boundary held because something kept checking that it held.

How it works¶

The suite is organised around the boundary rather than around features: for each supported input class, a case asserting acceptance and correct handling; for each out-of-domain class, a counterexample asserting clean rejection; for the output side, assertions that responses stay within the declared codomain and conformance rules across the full range of inputs, including edge and adversarial ones. It runs on every change, so the boundary is protected against silent regression. Run against the provider (as consumer-driven contracts) it also catches the case where the producer quietly widens its codomain.

Tuning parameters¶

Positive/negative balance — how many counterexamples versus happy-path cases. Skewing toward positives leaves the boundary's exclusion side unverified — the most common blind spot.
Boundary coverage — whether cases probe the exact edges (empty, max, off-by-one, malformed) or only comfortable interiors. Edge coverage catches real breaks but costs authoring effort.
Consumer-driven vs. provider-authored — whether real consumers' expectations define the contract, or the provider asserts its own. Consumer-driven catches unannounced codomain changes; provider-authored is easier to maintain.
Strictness of output assertions — exact match vs. schema-conformance vs. invariants only. Tighter assertions catch more drift but break on benign changes.

When it helps, and when it misleads¶

Its strength is turning a boundary into a regression-proof, always-on guarantee: once a counterexample exists, that particular way of violating the domain can never silently return. It also documents the boundary in runnable form — the tests are the specification of what's in and out. Its limits are the limits of every test suite: it verifies only the cases it contains, so an unwritten edge case is an unverified boundary, and green tests can breed false confidence that the whole boundary is safe. The classic misuse is writing tests after an implementation to certify whatever it already does — encoding the current (possibly wrong) behaviour as "the contract" rather than testing against an independently agreed boundary. The discipline is to derive cases from the declared domain and codomain (a Type Signature, Output Schema, or spec), lead with counterexamples, and treat coverage gaps as unverified boundary, not proven-safe.

How it implements the components¶

test_cases_and_counterexamples — its core output: the executable positive cases and boundary-violating counterexamples that make the boundary testable and teachable.
membership_criteria — encodes the input-membership rules as executable assertions, verifying that supported inputs are accepted and unsupported ones rejected.
output_conformance_rule — asserts across the input range that every output stays inside the declared codomain, catching a producer that silently exceeds it.

It verifies boundaries others author; it does not itself declare the input_domain/output_codomain (Type Signature, Output Schema), enforce them on live traffic (Input Validation Gate, Output Validation), or govern their revision (Scope Change Review).

Instantiates: Domain–Codomain Delimitation — the suite is how a declared boundary is proven to actually hold in the implementation.
Consumes: Type Signature and Output Schema supply the declared domain and codomain the cases are derived from.
Sibling mechanisms: Type Signature · Output Schema · Input Validation Gate · Output Validation · Eligibility Criteria · Model Applicability Card · Service Scope Statement · Clinical Indication Criteria · Scope Change Review · Unsupported Case Triage Workflow

References¶

[1] Consumer-driven contract testing (popularised by tools such as Pact) — the consumer's expectations of a provider's inputs and outputs are captured as a contract and verified against the provider in its own build. It is the standard way to catch a provider quietly widening its codomain before that change reaches production. ↩