Skip to content

Full-Factorial Joint-State Test

Method — instantiates Conjunctive Path Assurance

Runs every combination of the governing state variables against the system and checks each one for the combination that lets the whole route conduct.

Where a solver reasons about a model and a twin rehearses chosen scenarios, Full-Factorial Joint-State Test does the brute-force thing: it exercises every combination of the governing state variables against the real (or realistic) system and checks each for the forbidden outcome. Its defining move is exhaustiveness — no combination within the chosen variable set goes untested — so the guarantee it buys is unusually strong: if a dangerous conjunction lives among those variables, the test is certain to have hit it. The price is combinatorial: the number of runs is the product of the variables' levels, so the method is honest only about the small set of variables it can actually afford to cover, and ruthless about testing all of them.

Example

An elevator controller has four safety-relevant boolean inputs: door-sensor bypass active, fire-service mode engaged, overload switch tripped, leveling fault present. That is 2⁴ = 16 combinations, and the test harness drives all sixteen against the controller, checking each for the forbidden outcome — car moves with the doors open. Fifteen behave correctly. One does not: with the door-sensor bypass active and a leveling fault present at the same time — a conjunction neither input triggers on its own and that no one had thought to test jointly — the car creeps with the doors open. Because the test is exhaustive over these four inputs, the harness was guaranteed to reach that combination rather than lucky to. The reproducible failing case goes straight to engineering.

How it works

Choose the variables and the levels each can take; generate the full Cartesian product of them; drive every resulting combination through the system under the same end-to-end pass/fail oracle; and log which combinations conduct. There is no sampling and no judgement about which combinations "matter" — the coverage plan is simply all of them. What distinguishes it is exactly that completeness: the only way a dangerous combination escapes is if one of its variables was left out of the set, or its level was never tested — which is why the choice of variables carries the entire risk.

Tuning parameters

  • Variable and level selection — which inputs are included and how many levels each takes. This is the whole game: a variable left out is a combination never tested, and each variable added multiplies the run count.
  • Oracle strictness — how tightly "the route conducted" is defined. A loose oracle passes a marginal hazard; a strict one may flag benign edge behaviour.
  • Environmental fixing — which background conditions are held constant versus varied. Holding them fixed shrinks the matrix but can hide interactions with them.
  • Execution budget — real hardware runs versus simulation, and how much of the full product you can afford before you must drop to a reduced design.

When it helps, and when it misleads

Its strength is completeness within its chosen variables: every joint state is actually exercised, so no modeled combination is assumed safe — a guarantee sampling and hand-picked scenarios cannot give. Its failure mode is combinatorial explosion — the run count is the product of the levels, feasible only for a small variable set — and the moment a variable is dropped to stay affordable, untested combinations return; it also tests the combinations but not their timing or ordering unless that is itself encoded as a variable. The classic misuse is quietly shrinking the variable set to fit the budget and then claiming "we tested everything." The discipline is to choose the variables from the hazard analysis, state plainly what was excluded, and move to a reduced (t-wise) design deliberately rather than by dropping factors on the sly.[1]

How it implements the components

  • combinatorial_coverage_plan — its coverage plan is the maximal one: the full Cartesian product of the chosen variables, nothing skipped.
  • joint_state_variable_set — the variables and their levels are exactly what define the test matrix.
  • end_to_end_activation_oracle — each combination is judged by the same check of whether the complete route conducted.

It runs; it does not prove or time: the minimal_activating_conjunction_set a solver would isolate is Boolean SAT or SMT Path Search's and Model Checking and Reachability Analysis's, and the temporal_overlap_model for timing is Digital-Twin Hazard Rehearsal's; t-Wise Combinatorial Interaction Testing covers the same space with far fewer runs.

References

[1] A full factorial design exercises every combination of factor levels; its run count grows as the product of the levels — the combinatorial explosion — which is why exhaustive joint-state testing stays feasible only for a handful of variables and is deliberately traded for a t-wise subset beyond that. ↩