Selective Parameter Freezing¶
Isolation method — instantiates Context-Keyed Representation Switching
Write-protects the parameters that encode one context's map so that learning a different context cannot overwrite them, drawing the isolation boundary in parameter space.
When several context-maps live in one parameter set — a genuinely shared substrate — updating for context B can silently degrade context A's map, the failure known as catastrophic forgetting. Selective Parameter Freezing pins the parameters that carry A's map (and the shared-invariant core underneath all contexts) so that B's update is only allowed to move the parameters that are A-neutral. Its distinguishing idea is where the isolation boundary is drawn: not by giving each map its own address or its own snapshot, but by making the protected weights read-only during another context's update. The wall is in parameter space, and it is enforced at the moment of learning.
Example¶
A shared perception network must handle both daytime and night driving on the same weights. Fine-tuning on a fresh batch of night data reliably wrecks daytime performance — the night gradients drag the shared weights toward night and the daytime map is forgotten. Selective Parameter Freezing identifies the units that daytime performance depends on, and the low-level feature backbone common to both, and freezes them: their gradients are zeroed for the night update. Only a subset of higher-layer parameters is left free to adapt. The result is night performance that improves while daytime holds, because the daytime-critical and shared-invariant weights were write-protected exactly while the interfering update ran.
How it works¶
- Identify which parameters encode the map(s) to protect, which form the shared-invariant core, and which are freely adaptable.
- Freeze the protected and invariant parameters — zero their gradients or mark them read-only — for the duration of the other context's update.
- Update only the unfrozen subset.
- Refreeze along the new boundary as protection shifts to a different context.
Tuning parameters¶
- Freeze fraction — how much of the network is frozen. More frozen means stronger protection and less capacity to learn the new context — the plasticity–stability trade-off in one dial.
- Freeze granularity — whole layers versus individual units or weights. Finer preserves capacity but is far harder to target correctly.
- Hard versus soft freeze — a strict read-only freeze, or a penalty that merely resists moving protected weights. Soft allows negotiated change; hard guarantees zero drift.
- Boundary source — how "protected" is decided: measured parameter importance, task masks, or explicit layer assignment.
When it helps, and when it misleads¶
Its strength is that it lets several context-maps coexist on one substrate without keeping a full copy per context, directly countering catastrophic forgetting[1] and protecting the shared-invariant layer every context relies on. Its failure modes are all about the boundary: freeze too much and the new context cannot be learned at all (it underfits); freeze the wrong parameters and you protect the unimportant while the important ones drift; and a hard freeze cannot accommodate a context that legitimately needs to revise shared structure. The classic misuse is freezing broadly as a blunt "don't break anything" reflex, which starves the model of the plasticity it needs and hides that the substrate was over-shared to begin with. The discipline is to base the freeze boundary on measured parameter importance for each protected context, and to revisit it — a static freeze fossilizes yesterday's partition of the maps.
How it implements the components¶
Selective Parameter Freezing fills the on-substrate isolation subset — keeping shared-parameter maps from overwriting each other:
representation_isolation_boundary— freezing is the boundary, drawn in parameter space: frozen weights are the wall one context's update cannot cross into another's map.shared_invariant_layer— it explicitly protects the invariant core valid across all contexts by freezing it while only the context-specific parameters adapt.
It does not keep maps apart by naming or addressing them separately (that's the Context-Tagged Namespace Partition), preserve a full snapshot for later restore (that's Per-Context Model Checkpoint), or gate the promotion of an updated map (that's Shadow-Map Evaluation).
Related¶
- Instantiates: Context-Keyed Representation Switching — protects inactive maps from interference when they must share a parameter set.
- Sibling mechanisms: Context-Tagged Namespace Partition · Shared Backbone with Context Adapters · Per-Context Model Checkpoint · Shadow-Map Evaluation · Cross-Map Interference Regression Suite
Editorial Notes¶
Form Classification¶
Form family: Intervention, Treatment & Transformation
Rationale: Selective Parameter Freezing operates by directly freezes protected parameters while allowing selected parameters to adapt. That concrete deployed or enacted form is Intervention, Treatment & Transformation under the frozen taxonomy.
Nearest alternative: Control, Automation & Runtime — Although Control, Automation & Runtime can support this mechanism, the frozen evidence makes its operative form the act that directly freezes protected parameters while allowing selected parameters to adapt; the alternative is therefore secondary rather than defining.
Review outcome: Adjudicated after independent review; high confidence.
Origin Attribution¶
Primary origin: Computer Science & Software Engineering
Origin pattern: Single lineage
Present-day reach: Specialized
Rationale: Freezing subsets of learned parameters to prevent catastrophic interference is a machine-learning transfer and continual-learning technique.
Related originating lineages:
- Cognitive Science — Stability-plasticity research supplies the conceptual problem of retaining old mappings while learning new contexts.
- Engineering & Design — Engineering design, reliability, and systems-safety practice supplies a parallel or contributing lineage for the mechanism's defining operation: write-protects the parameters that encode one context's map so that learning a different context cannot overwrite them, drawing the isolation boundary in parameter space.
- Statistics & Experimental Design — Constrained estimation provides an older mathematical analogue for fixing some parameters while fitting others.
Review resolution: The blind reviewers agree that computer_science is the primary origin and differ only on alternate origin disagreement, encyclopedia synthesis disagreement. I preserve every independently explained alternate from both records rather than imposing a numeric cap. I retain single_lineage because the combined record shows one traceable formative lineage. The broader reach of specialized records portability separately from historical provenance, and encyclopedia_synthesis=true preserves the affirmative synthesis judgment where either reviewer identified one.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
Parameter freezing, Context-Tagged Namespace Partition (isolation by address), and Shared Backbone with Context Adapters (isolation by module) are three ways to stop maps colliding on a shared substrate. Freezing is the right tool specifically when the maps must share one parameter set and cannot be cleanly separated into namespaces or modules — it is the finest-grained and the most delicate of the three to target.
References¶
[1] Kirkpatrick, J., et al. "Overcoming Catastrophic Forgetting in Neural Networks". Proceedings of the National Academy of Sciences 114(13), 3521–3526 (2017). Shows that elastic weight consolidation can retain several sequentially learned tasks in one network by selectively protecting weights important to earlier tasks. registry ↩