Policy Symmetry Test¶
Rule test — instantiates Symmetry-Based Fairness
Interrogates a rule as written or as coded — would it hand equivalent cases different treatment when only an irrelevant feature changes? — to catch asymmetry and smuggled values in the policy itself, before a single case is decided.
Policy Symmetry Test works on the rule, not on the pile of decisions the rule has produced. It takes a policy — a written eligibility standard, a scoring rubric, a coded decision procedure — and probes it with transformations that ought to be irrelevant: change the reviewer, the presentation order, an identity label, the applicant's location. If the rule's output moves when only an irrelevant feature moved, the rule itself is asymmetric, and it will manufacture unequal treatment no matter how conscientiously anyone applies it. Its defining move is that it is a priori: it can convict a policy of unfairness before that policy has touched a real person, because the flaw is in the text or the code, not in how carefully a human runs it.
Example¶
A city rewrites the rubric it uses to award public-works contracts and asks whether the rubric treats equivalent bids equivalently. A Policy Symmetry Test takes two bids that are identical on every performance-relevant fact — price, timeline, safety record, staffing — and varies only features that should not matter: which of the two bidders is the current incumbent, which analyst scores it first, whether the proposal was submitted as a PDF or through the portal. Tracing the rubric's arithmetic, the test finds a clause that grants five points for "prior successful engagements with the city." On paper it reads as neutral experience; under the swap it is exposed as a pure incumbency bonus with no tie to performance — a thumb on the scale for whoever already holds the contract. That is a value (protect incumbents, minimize switching risk) smuggled into a rule that claimed to score only capability. The test's output is not a verdict on any one award but a redline on the rubric: this clause breaks symmetry, name the value or remove it.
How it works¶
- Recover the rule's implicit equivalence relation. Read out which facts the policy claims to act on; those, and only those, are supposed to move the outcome.
- Enumerate irrelevant transformations. List the changes that must leave treatment untouched — reviewer identity, order, label, location, format — drawn from where this policy is likely to leak.
- Trace or probe. For a short rule, reason through the text; for a coded one, run matched synthetic pairs through the implementation that differ only on an irrelevant feature.
- Adjudicate each break. Where the output moves, ask whether the moving feature is a genuine relevant difference or a value quietly embedded — and force it to be stated either way.
Tuning parameters¶
- Transformation set — how many irrelevant features you probe. A wider set catches more leaks but costs analysis time; narrow it to the ones this policy plausibly leaks through.
- Symbolic vs. empirical probing — reading the rule versus running paired synthetic inputs through the live implementation. Empirical probing catches gaps between the written rule and the deployed code that reading never will.
- Value-surfacing depth — whether you stop at "this feature moves treatment" or push on to name the value it encodes. Deeper surfacing is what turns a symmetry bug into a governable policy choice.
- Scope — a single contested clause versus the whole policy. Whole-policy sweeps are thorough but slow; clause-level tests are fast triage.
When it helps, and when it misleads¶
Its strength is prevention: it catches an asymmetric rule before that rule produces one unfair decision, and it drags smuggled values into daylight where they can be argued about rather than absorbed. A rule that passes is one you can defend on its face.
Its failure mode is that a clean rule can still be applied dirtily. Policy Symmetry Test sees the text and the code; it cannot see a reviewer who reads the neutral rule and then quietly bends it in discretionary application. A policy that is symmetric on paper and skewed in practice will sail through this test and fail in the field — so treating a pass as proof of fairness is the classic misuse. The guarding discipline is to treat a symmetric rule as necessary but not sufficient, and to pair the rule test with an outcome audit that measures what the policy actually did. The formal principle of equality it enforces — treat like cases alike — is famously silent about which likenesses count,[n1] which is exactly why the test's real work is forcing the rule to declare them.
How it implements the components¶
Policy Symmetry Test realizes the rule-inspection face of the archetype — the components that live in the policy itself rather than in the case record:
equivalence_rule— it makes the rule's implicit "these cases are the same" relation explicit and checkable, which is the only way to test whether the rule honors it.equal_treatment_rule— it verifies that the treatment the rule specifies stays invariant across inputs that differ only on irrelevant features.value_priority_statement— by asking why a feature moves treatment, it surfaces the contested value the rule embeds and forces it to be named rather than smuggled.
It does not measure realized outcomes across a population (case_set_scope, audit_trail) — that is [Consistency Audit], which reads decisions after the fact rather than the rule before it; nor does it line a case up against decided precedents (precedent_reference) — that is [Precedent Analysis].
Related¶
- Instantiates: Symmetry-Based Fairness — Policy Symmetry Test supplies the rule-level assurance the archetype's equal-treatment invariant depends on.
- Sibling mechanisms: Consistency Audit · Equal-Treatment Checklist · Anti-Discrimination Check · Precedent Analysis · Classification Fairness Review · Exception Register
Editorial Notes¶
Form Classification¶
Form family: Experiment, Test & Rehearsal
Rationale: The mechanism deliberately changes only irrelevant features in traced or synthetic matched cases and checks whether the policy's treatment remains invariant.
Nearest alternative: Analysis, Modeling & Optimization — Reasoning can analyze a short rule, but controlled minimal-pair probing is the defining validation operation.
Review outcome: Adjudicated after independent review; high confidence.
Origin Attribution¶
Primary origin: Law & Governance
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Testing equal treatment of relevantly equivalent cases belongs to legal equality and anti-discrimination doctrine.
Related originating lineages:
- Philosophy — Philosophy contributes the formal symmetry test and the distinction between relevant and irrelevant features.
Review resolution: Both blind reviewers agree that law governance is the primary origin. Reconciliation resolves encyclopedia synthesis disagreement. Formative alternate lineages are retained as philosophy; later breadth of use is recorded separately as domain_reach=multi_domain, while origin_mode=cross_disciplinary_synthesis describes the relationship among origin lineages.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
[n1] The formal principle of equality — treat like cases alike, and unlike cases differently in proportion to their relevant differences — is traced to Aristotle's Nicomachean Ethics. It is deliberately empty about which differences are relevant, and that emptiness is precisely the gap a policy symmetry test forces a rule to fill. ↩