Skip to content

Negative Case and Malformed Corpus

Test-data artifact — instantiates Asymmetric Interface Tolerance Calibration

A curated collection of known-bad inputs, each paired with the verdict and diagnostic it should provoke, held as the fixed yardstick for what the interface must refuse.

A Negative Case and Malformed Corpus is a curated, version-controlled collection of known-bad inputs, each paired with the verdict it should provoke — reject, or tolerate-and-repair-to-X — and the diagnostic the producer should get. Its defining move is to be the fixed yardstick for what the interface must refuse: where positive tests pin down what must be accepted, this pins down what must be rejected, and how. It differs from Fuzz Testing Against Acceptance Boundary in kind: the fuzzer generates endless new inputs, while the corpus is a stable, hand-curated set that every build is re-checked against, so a boundary bug, once fixed, can never silently return.

Example

A team building a TLS certificate validator keeps a corpus of malformed and adversarial certs: one with a valid-looking but expired date, one with a mismatched signature, one with an embedded null byte in the common name (the classic name-confusion trick), one with a pathological chain depth. Each entry records the expected outcome — reject, with a specific reason code. Every build runs the whole corpus. If a refactor makes the validator start accepting the null-byte cert, the corpus turns red immediately. When a new attack variant appears in the wild, it becomes a new corpus entry, so the fix is permanent. Setup to outcome: a growing, regression-proof definition of exactly what the validator must never accept, and why.

How it works

  • Each case is paired with its expected verdict and diagnostic, so the corpus tests not just "is it rejected" but "is it rejected for the right reason, with the right message."
  • It is stable and versioned — the same cases run every build — which makes it a regression anchor rather than a one-off probe.
  • Its cases are harvested from reality: past incidents, fuzzer findings, real-world malformations, not only imagined ones.
  • It grows monotonically — every new field-observed bad input becomes a permanent entry that can never regress unnoticed.

Tuning parameters

  • Coverage taxonomy — which malformation classes to represent (truncation, encoding tricks, type confusion, out-of-range, injection). Broad coverage catches more but costs curation.
  • Verdict granularity — whether a case pins just accept/reject or the exact diagnostic and error code. Pinning the diagnostic guards message quality but makes the corpus brittle to wording changes.
  • Provenance mix — hand-authored versus harvested-from-telemetry versus fuzzer-minimized. Harvested and minimized cases stay grounded in inputs that really occurred.
  • Freshness policy — how a newly observed malformation gets promoted into the corpus, and how fast. A corpus that does not ingest new field cases slowly goes stale.
  • Prune versus pin — whether obsolete cases are removed or kept as permanent regression anchors. Pinning bloats the suite but guarantees old bugs stay dead.

When it helps, and when it misleads

Its strength is that it makes rejection behavior durable and precise — a fixed contract of what must be refused and why, so hard-won boundary fixes do not silently regress and diagnostics stay useful. The classic misuse is the corpus that rots: never updated as new malformations appear, so a green run gives false confidence about a boundary reality has already moved past. A close cousin is curating only cases the interface already handles (survivorship), which tests nothing. The discipline that keeps it honest is to feed the corpus continuously from the Malformation Rate Dashboard, the quarantine queue, and fuzzer findings, and to treat every field incident as a mandatory new entry — the corpus is only as honest as its most recent addition.

How it implements the components

The corpus fills the ground-truth-data side of the archetype's machinery — the durable specification the runners operate on:

  • conformance_and_fuzz_oracle — it supplies the ground-truth cases and their expected verdicts the oracle judges against, and the seeds a fuzzer mutates; it is the oracle's memory.
  • rejection_and_diagnostic_policy — by pinning the exact verdict and message per case, it specifies what the rejection-and-diagnostic policy must do, turning that policy from prose into checkable examples.

It is data, not a runner: it does not generate new inputs — that is Fuzz Testing Against Acceptance Boundary — nor check positive conformance (Contract Test Suite), nor monitor live rates, which is the Malformation Rate Dashboard.

Notes

The corpus is a shared artifact several mechanisms operate on: the fuzzer seeds from it, the contract gate and test suite assert against it, and the dashboard and quarantine queue feed it. Its value scales with how many of them draw on it and, above all, with how fresh it is kept — curation is a standing job, not a setup task.