Skip to content

Supervised Representation Learning

A learned model — instantiates Model-Guided Signal Separation

Learns a separator from labeled examples — fitting a representation that keeps target-linked variation and discards the rest, instead of deriving it from a known model of the mixture.

Where the other separators derive their operator from an explicit model of how sources combine — a convolution kernel, a mixing matrix, a noise spectrum — Supervised Representation Learning has no such model. It substitutes labeled examples for one: given many pairs of (observation, known target), it fits a mapping that predicts the target and, in doing so, learns a representation in which target-linked variation survives and nuisance variation collapses. What makes it this mechanism and not its siblings is exactly that inversion — the labels operationally define the target, and the separator is estimated from data rather than reasoned from physics. That is its power where the mixing is too complex to write down, and its hazard where the labels quietly encode the wrong thing.

Example

A manufacturer inspects welds by radiography, and wants to pull genuine crack and porosity signatures out of the normal speckle and grain texture of an X-ray. The physics of the mixture is hopeless to model, but there are years of images with defects outlined by inspectors. A model is trained on those pairs to output a per-pixel defect map. It learns a representation that responds to the fine linear discontinuity of a crack while staying flat over ordinary weld texture — separating the target from the nuisance not because anyone specified the difference, but because the labeled examples did. On new radiographs it highlights suspect regions for a human to confirm. The catch surfaces only later: if most training cracks happened to be photographed on one machine, the model may be keying on that machine's texture, not the crack.

How it works

The method's distinguishing move is that no forward or mixing model is written at all. A function from observation to target is fitted directly on labeled pairs, and the internal representation is pushed to be predictive of the label while invariant to nuisance — invariance supplied through data augmentation, architecture, or explicitly labeled nuisance. The whole enterprise then stands on generalization: the separator is only as good as the labeled distribution it saw, and its behavior off that distribution is unwarranted.

Tuning parameters

  • Label quantity and quality — more and cleaner labels sharpen the target's definition; scarce or noisy labels cap how finely target and nuisance can be told apart.
  • Built-in invariances — the augmentations and architecture that declare which variation is nuisance; too few and the model latches onto confounds, too many and it erases real target variation.
  • Capacity vs. regularization — larger models capture subtler separations but memorize spurious cues; regularization trades some sharpness for robustness.
  • Supervision target — what the label actually encodes (mere presence, a mask, a clean reconstruction); it fixes what "the signal" even is.

When it helps, and when it misleads

It shines exactly where the mixture is unknown, nonlinear, or too tangled to model but labeled examples are plentiful — it will separate signals no closed-form operator could. Its failure mode is shortcut learning: the model latches onto a nuisance feature that happens to correlate with the label in the training set, so its output looks clean and then collapses under distribution shift.[1] Its classic misuse is the signal-separation version of running a model backwards — tuning against the evaluation data until the held-back examples light up, mistaking a fitted artifact for the target. The discipline that guards against both is to freeze the target definition before fitting and to test the separator on genuinely out-of-distribution data and injected signals, never on the sample it was tuned to.

How it implements the components

  • separation_operator_specification — the fitted mapping from observation to target is the separation operator, learned rather than derived.
  • calibration_reference — the labeled examples are the ground-truth reference the whole fit is anchored to.
  • target_signal_specification — the labels operationally define what counts as the target.
  • domain_constraint — the architecture and built-in invariances encode which variation is nuisance and which is signal.

It does not quantify its own estimate uncertainty (that is Kalman or Particle Filter) or verify its output — the honest test of a learned separator is external, via Held-Out Sample Test and Signal Injection–Recovery Test.

Notes

Because the labels define the target, this mechanism can only separate what the labeling scheme already names — it cannot surface an unlabeled or unsuspected source. When the interesting signal has no labels yet, the template-free counterpart is Blind Source Separation, which finds sources from statistical structure alone.

References

[1] Shortcut learning (a form of spurious correlation) — a model exploits a nuisance feature that predicts the label in the training data but is not part of the true signal, so accuracy on the training distribution masks a separator that fails when the nuisance decouples from the target. It is the standard reason a learned extractor that looks clean in validation degrades in deployment. ↩