Boundary Condition Matrix¶
Analytic artifact — instantiates Counterexample Search
Lays a rule's operating dimensions on a grid and marks each cell tested-pass, tested-fail, or untested, so the coverage gaps become as visible as the found failures.
Exceptions found one at a time are easy to lose track of, and the cases nobody looked at leave no trace at all. Boundary Condition Matrix is the artifact that fixes both problems. It is not an act of searching or revising — it is a persistent grid. It lays the claim's operating dimensions on axes and marks, cell by cell, where the rule has been challenged and how it fared, and — its whole point — leaves the untested cells visibly blank. It is the standing inventory of negative cases and the running coverage note in one object: it accumulates found failures across uses and, at the same time, makes the map of unexamined territory impossible to ignore.
Example¶
A team maintains the rule "our address parser correctly normalizes any valid postal address." A Boundary Condition Matrix lays out axes: country (US, JP, DE, IN, …) × script (Latin, CJK, …) × field completeness (full, missing postcode, …) × source (web form, OCR, API). Each cell carries a state — tested-pass, tested-fail (with the stored failing input attached), or untested.
The filled grid says two things at once. A cluster of tested-fail cells at JP × CJK × OCR marks a real boundary where the rule breaks and holds the negative cases that prove it. And a large block of untested cells — India, missing-postcode — shows exactly where the team's current confidence is unearned: not "the parser works there," but "no one has checked." Each release adds cases and repaints cells, so the inventory grows and the blank region shrinks visibly over time. The matrix never decides what to do about a broken cell — it makes sure the break, and the gap, are both on the table.
How it works¶
- Choose the axes. Pick the dimensions along which the rule is most likely to vary — the ones where a boundary would live.
- Populate from cases. Each challenged cell records its outcome and, on failure, stores the negative case itself.
- Expose the gaps. Untested cells are the headline output: they turn "no counterexample found" back into the honest "no one has looked here yet."
- Persist and accrete. The matrix is reused release over release, not discarded — it is the running inventory, not a one-shot report.
Tuning parameters¶
- Axis granularity — how many dimensions and how finely sliced. Finer grids catch more boundaries but explode combinatorially in cells to fill.
- Cell vocabulary — plain pass/fail/untested, or richer states (partial, flaky, deprecated). Richer captures nuance but costs upkeep.
- Coverage target — what fraction of cells must be exercised before the rule is trusted. Higher is safer and costlier.
- Persistence scope — per-release snapshots vs. a cumulative board. Cumulative shows history but can mask regressions unless cells are dated.
When it helps, and when it misleads¶
Its strength is converting scattered exception-hunting into a visible coverage map, and defusing the archetype's most seductive error — reading "we found no counterexample" as "there is none" — by literally rendering the blank cells that show where confidence is unbacked.
Its failure mode is false comfort from a tidy grid: a matrix that looks reassuringly full while its axes simply omit the dimension that actually matters — the untested axis nobody thought to draw. The classic misuse is presenting a dense green board as proof of universality when the real failure mode lives off-grid. The guarding discipline is to treat the matrix as a coverage note, never a completeness proof, and to keep asking what axis is missing — you can only claim coverage over dimensions you enumerated, the core caution of combinatorial coverage testing.[n1]
How it implements the components¶
negative_case_inventory— it is the persistent, reusable store of negative cases, organized on the grid for recurring use.search_coverage_note— its untested cells are a standing coverage note, keeping absence of found counterexamples from hardening into false certainty.
It catalogs and displays but never acts on the cases — turning a filled cell into a narrowed rule or a lowered confidence level (scope_revision, confidence_update) is Negative Case Analysis's job, its nearest twin.
Related¶
- Instantiates: Counterexample Search — it is the coverage artifact that keeps the search honest over time.
- Consumes: Exception Search supplies the found cases that populate the cells; Falsification Check supplies the scope that defines the axes.
- Sibling mechanisms: Falsification Check · Exception Search · Adversarial Example Generation · Negative Case Analysis · Proof by Counterexample
Editorial Notes¶
Form Classification¶
Form family: Representation, Specification & Plan
Rationale: The mechanism persists a grid of a rule's operating dimensions with tested-pass, tested-fail, and untested status and associated negative cases, so its operative form is a boundary-coverage representation.
Nearest alternative: Assessment, Review & Assurance — Individual cases are assessed to populate cells, but the matrix's continuing value is the accumulated map of failures and untested regions.
Review outcome: Adjudicated after independent review; high confidence.
Origin Attribution¶
Primary origin: Engineering & Design
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Mapping tested-pass, tested-fail, and untested cells over an operating envelope is a verification and validation artifact from engineering.
Related originating lineages:
- Computer Science & Software Engineering — Combinatorial software testing makes coverage over input and configuration dimensions explicit and measurable.
- Statistics & Experimental Design — Factorial design and coverage reasoning systematically choose condition combinations and distinguish tested from untested regions.
Review resolution: A pass/fail/untested matrix over an operating envelope is an engineering verification-and-validation artifact. NIST's combinatorial-testing work formalizes the supporting matrix of parameter values and state-space coverage, while experimental design supplies systematic variation; engineering remains primary because the page uses those methods to map validated operating conditions.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Researched adjudication after independent review; high confidence.
Sources consulted:
- NIST — Combinatorial Methods for Trust and Assurance: Coverage Measurement — NIST defines test sets as matrices of parameter values and measures which combinations of the input or configuration state space have and have not been covered.
Notes¶
[n1] Combinatorial (coverage-array) testing — the discipline of enumerating the input dimensions and their interactions so that coverage is measured against an explicit space rather than assumed. Its central lesson is exactly the matrix's caveat: you can only claim to have covered the dimensions you put on the grid. ↩