Skip to content

Witness Validation Test

Witness validation protocol — instantiates Target-Complete Mapping Design

Executes a claimed witness under realistic conditions to confirm it actually reaches its target — turning a recorded link into dated evidence and unmasking phantom witnesses that exist only on paper.

A link on the coverage map is a claim, not a proof. Witness Validation Test takes a specific claimed witness — a source recorded as covering a target — and actually exercises it end-to-end under the conditions that define the target, confirming the coverage relation genuinely holds and recording the result as dated coverage evidence. Its defining move is to attack the phantom witness: a link present on the map but false in reality. It demands that a witness perform, not merely exist, and it stamps the outcome with a version and date so coverage can expire rather than stay green forever.

Example

A company's continuity plan lists, for each critical service, a documented recovery runbook — so on paper every service has a witness. Witness validation runs a restore drill, a scheduled "game day": it takes the payments-service runbook and actually executes it against a clean environment.[1] The runbook references a nightly backup that turns out to be 0 bytes, and a failover step that requires the very control plane a real disaster would have taken down. The witness is a phantom — recorded, but it does not reach the target. The outcome is that the runbook is marked invalid with dated evidence, the payments target reverts to uncovered, and the fix is re-drilled; services whose drills pass receive a timestamped evidence record that expires and must be re-drilled on cadence.

How it works

What distinguishes this protocol is execution under real conditions plus evidence with an expiry, not a paper inspection. It defines the validity rule a witness must satisfy — same target version, authorized, feasible under declared conditions, current, sufficiently capable — then runs the witness against those conditions rather than reading its documentation. A witness that fails any clause flips its target to uncovered; a pass is recorded as evidence bound to a version and date, so the record lapses and forces re-validation on cadence instead of standing green indefinitely.

Tuning parameters

  • Validity depth — documentary attestation vs inspection vs simulation vs full end-to-end execution. Deeper testing catches more phantoms and costs more.
  • Condition realism — sandbox vs production-like vs live failure injection. The closer to the real target case, the more trustworthy the pass.
  • Evidence lifetime — how long a pass stays valid before it expires and the witness must be re-tested.
  • Pass-criteria strictness — which validity clauses are mandatory versus advisory for a target to count as covered.
  • Sampling vs census — validate every witness or a risk-weighted sample; a census is thorough but expensive.

When it helps, and when it misleads

Its strength is being the direct defense against the archetype's most dangerous failure — the recorded-but-false witness — and its dated evidence stops coverage from silently going stale. Its failure mode is that a test only validates under the conditions it chose: a shallow or unrealistic drill passes a witness that would fail in the real event, and full end-to-end validation is expensive enough to tempt teams toward attestation that proves little. The classic misuse is accepting a documentary sign-off in place of an actual drill and calling the target covered. The discipline that guards against it is to execute under conditions close to the real target case, and to let evidence expire so a witness cannot coast on a years-old pass.

How it implements the components

Witness Validation Test realizes the proof-of-coverage slice of the archetype — the components that make a witness credible:

  • witness_validity_rule — it defines and applies the pass/fail criteria a claimed witness must meet before its target may be marked covered.
  • coverage_evidence_record — it produces the dated, version-bound evidence that a witness actually holds, so coverage can expire and be re-proven.

It does not create a witness where none exists (that's Preimage Witness Generator), bind a test to the requirement it verifies (Test-Case-to-Requirement Linker), or test resilience to a witness's removal (Redundancy N+1 Check).

References

[1] A widely-held operations principle holds that an untested backup is not a backup; scheduled "game day" restore and failover drills exist because recovery witnesses that are never exercised routinely fail when finally needed. The test converts a paper witness into a demonstrated one — and lets the demonstration expire. ↩