Skip to content

Feature Ablation or Occlusion Test

Diagnostic intervention — instantiates Shortcut-Reliance Mitigation

Masks, removes, or permutes a suspected cue while holding everything else fixed, and reads the drop in performance as the model's reliance on that exact cue.

Once you suspect a specific crutch, you want to know two things: is the model actually using it, and how much. Feature Ablation or Occlusion Test answers both with a controlled intervention on the input itself. It takes one named cue — a background region, a metadata field, a header token, a color channel — masks or removes or permutes it while holding the rest of the input constant, and re-scores. The change in performance is a direct, quantified attribution: a large drop means the model was leaning on that cue; near-perfect accuracy from the cue alone means it was leaning on almost nothing else. Its defining move, and what separates it from a stress test, is surgical locality — it edits the contents of the input to isolate a single feature's causal contribution, rather than swapping the whole surrounding context. That locality is what turns "the model is brittle" into "the model gets 0.9 of its score from the timestamp."

Example

A camera-trap classifier that tells deer from wild boar scores an illustrative 96% on its validation set. A reviewer suspects habitat, not animal, is doing the work — boar were photographed mostly in dense undergrowth, deer mostly in open clearings. Two ablations settle it. First, occlude the animal and leave the background: accuracy stays around 89%, which should be impossible if the model were reading the animal. Second, occlude the background (inpaint it to neutral) and leave the animal sharp and centered: accuracy falls to about 61%. The reliance score is unambiguous — the classifier is mostly a habitat detector wearing an animal-classifier's label, and it will fail the moment a boar wanders into a clearing. The test does not fix anything and does not decide whether to ship; it hands a specific, sized finding to remediation and to the model card.

How it works

What distinguishes it is the paired, single-variable intervention. You draw a cue from the shortcut inventory, choose an intervention (mask, blur, remove, permute across examples, or substitute a counterfactual value), and apply it to that cue only, keeping every other feature fixed, so the score change is attributable to that one edit and nothing else. Two complementary readings matter: performance with the cue removed (how much the model loses without it) and performance from the cue alone (how much it can do with only the cue). Run across the inventory, this yields a ranked reliance score per suspected feature — the quantitative core the rest of the appraisal reasons over.

Tuning parameters

  • Intervention type — masking, permutation, deletion, or counterfactual substitution. Permutation preserves the marginal distribution; crude masking can create inputs unlike anything in training and mislead.
  • Replacement baseline — what you put where the cue was: zeros, dataset mean, blur, or realistic inpainting. An unrealistic fill makes the input itself the problem and inflates the apparent drop.
  • Granularity — ablate one feature or a correlated group at once. Removing a single member of a correlated cluster lets the model route around it and hides the reliance.
  • Paired vs. marginal — measure each cue in isolation, or leave-one-group-out against a full-input baseline. Isolation attributes; grouping catches redundancy.
  • Aggregation — how per-example score changes roll up into the headline reliance number (mean, worst-case, or distribution).

When it helps, and when it misleads

Its strength is precision and cheapness: no retraining, and an answer of the form "this exact cue accounts for this much of the score," which is what remediation and documentation actually need. It is the most direct way to convert a suspicion into a measured reliance.

Its sharpest failure mode is correlated features. If the suspected cue has a redundant partner, removing it alone lets the model lean on the partner and the test under-reports reliance — the classic suppressor problem. The opposite error is an out-of-distribution artifact: a clumsy mask produces an input so unnatural that performance collapses for reasons unrelated to reliance, over-reporting it. And like any attribution, an occlusion map can be run backwards — the cue and mask chosen after the fact to produce the saliency picture someone wanted.[^ablation] The discipline is realistic replacements, ablating correlated cues as a group, and corroborating a positive finding with an invariance probe rather than trusting a single map.

How it implements the components

Feature Ablation or Occlusion Test realizes the measurement side of the archetype — turning a named suspicion into a sized number:

  • feature_ablation_and_occlusion_plan — it specifies which cues to intervene on, by what method, against which baseline: the plan that makes the test repeatable and attributable.
  • shortcut_reliance_score — the performance delta under intervention is the reliance score, per cue and aggregated, that the acceptance gate and the model card consume.

It does not construct the shifted operating environments or set the ship bar — that is Domain-Shift Stress Test's matrix and gate — and it is a behavioral probe, not the human judgment of whether the features are semantically legitimate, which belongs to Causal Feature Review Panel.