Reward Prediction Error¶
Core Idea¶
A reward prediction error is the structural pattern in which a system learns not from raw outcomes but from the gap between expected and received outcomes, and uses the sign and size of that gap, rather than the outcome itself, to update its model. Outcomes that match expectation generate no error and produce no learning; outcomes that exceed expectation produce a positive error and reinforce whatever predicted them; outcomes that fall short produce a negative error and weaken those predictors. The essential commitment is that the system carries a prediction (an expectation, forecast, or value estimate), receives a signal (an outcome, reward, or measurement), and computes a scalar error (signal minus prediction) that serves as the teaching signal for whatever process updates the predictor.
Every instance specifies four parameters. There is (1) the predictor — the model that issues expectations; (2) the prediction — its output on a particular trial; (3) the observed outcome; and (4) the learning rate — how strongly the error updates the predictor. The error is the load-bearing currency of learning: a system without prediction errors keeps no record of surprise and does not improve. The pattern lets a reasoner ask crisp questions that raw-outcome accounts cannot — whose prediction error, against what predictor, with what learning rate.
The pattern is the dual of outcome-only learning. A Pavlovian organism that salivates when food arrives is responding to the stimulus, not to its mismatch with expectation; a prediction-error learner that already expected the food learns nothing from its arrival, while unexpected food (positive error) or unexpectedly absent food (negative error) is what teaches. The discipline is sharp: surprise is the teacher, and the expected is silent. A structural consequence is the baseline-shift phenomenon — as the predictor improves, the errors shrink and learning slows of its own accord, so the absence of further error signals that the system has reached its current ceiling, not that effort has failed.
How would you explain it like I'm…
The Surprise Teacher
Better Or Worse Than Expected
Surprise Is The Signal
Structural Signature¶
the predictor issuing expectations — the prediction on a given trial — the observed signal — the signed scalar error (signal minus prediction) — the learning rate scaling the update — the baseline-shift consequence (error shrinks as the predictor improves)
A system runs on reward prediction error when each of the following holds:
- A predictor. A model — value function, filter state estimate, consensus forecast, generative model — that issues expectations about outcomes.
- A prediction. The predictor's output on a particular trial, against which the outcome will be compared.
- An observed signal. An outcome, reward, or measurement arrives through some channel.
- A signed scalar error. The system computes signal minus prediction, retaining sign and magnitude; matched outcomes give zero error, positive errors reinforce predictors, negative errors weaken them. The error, not the outcome, is the teaching signal — surprise teaches, the expected is silent.
- A learning rate. A parameter governing how strongly the error updates the predictor, ideally matched to the signal's noise.
- A baseline-shift dynamic. As the predictor improves, errors shrink and learning slows of its own accord, so absent error signals an approaching ceiling, not failed effort.
The components compose the dual of outcome-only learning: a predictor, a signal, a signed error against it, and an update scaled by a learning rate — letting a reasoner ask whose error, against what predictor, with what rate, and diagnose failure by isolating which of the four components is broken. A structural refinement is sign asymmetry: positive and negative errors often update different pathways.
What It Is Not¶
- Not predictive coding.
predictive_codingis the architecture — a hierarchy passing predictions down and residuals up to minimize error across levels. Reward prediction error is the scalar teaching signal (signal minus prediction) itself; predictive coding is one system built from prediction errors, RPE is the error any such system uses. - Not reinforcement.
reinforcementstrengthens an action via its consequence. RPE specifies what drives that strengthening: not the raw reward but the signed gap between received and expected reward — the prediction-error decomposition inside reinforcement, not the whole selection engine. - Not Bayesian updating.
bayesian_updatingrevises a full probability distribution via likelihoods. RPE is a scalar point-estimate correction (a learning-rate-scaled error); the Kalman filter is the special Gaussian case where the two coincide, but RPE carries no posterior, only a signed residual. - Not feedback.
feedbackroutes output back to modify input at runtime. RPE is a learning signal that updates a predictor's model; a feedback loop with no predictor and no expectation computes no prediction error. - Not the outcome. The teaching signal is the surprise, not the reward level — a large but fully anticipated outcome teaches nothing (zero error), and an unexpectedly absent reward teaches via a negative error. Conflating outcome magnitude with learning is the core error the prime exists to dispel.
- Common misclassification. Reading persistent zero error as success. It can mean mastery or a sealed-off model no longer receiving informative input — distinguishable only by perturbing with a novel probe and watching whether an error appears.
Broad Use¶
- Computational neuroscience: midbrain dopamine neurons fire on unpredicted reward and pause below baseline when a predicted reward fails to arrive — the canonical biological prediction-error signal, used to update action values.
- Reinforcement learning: the temporal-difference error drives every TD-family algorithm, from Q-learning and actor-critic to deep-RL agents; without it the value function does not update.
- Bayesian filtering and forecasting: the innovation in a Kalman filter — observation minus predicted observation — is exactly a prediction error and is the only term that updates the posterior; ARIMA and exponential smoothing use the same error-driven update.
- Behavioural economics and finance: earnings surprise — actual minus consensus — moves asset prices, not the earnings level; pre-announced or anticipated earnings move nothing, and the unexpected component carries the signal.
- Education and feedback design: learners progress most when problems sit at the edge of expectation — too-easy problems produce no error, too-hard problems produce errors too large to be informative, and the productive zone is calibrated surprise.
- Forecasting and predictive coding: model skill is the residual against verification, and cortical hierarchies send only the prediction error upward while sending predictions downward, so perception is the suppression of prediction error.
Clarity¶
The pattern makes a critical distinction visible: there is a signal and there is a prediction, and learning is driven by neither alone but by their signed difference. Many naive theories conflate "more outcome" with "more learning"; a prediction-error account makes explicit that the same outcome can teach a lot, a little, or nothing depending entirely on what the learner already expected. This relocates the explanation of learning from the magnitude of the outcome to the magnitude of the surprise, which is a different and more predictive quantity.
Naming the four parameters — predictor, prediction, outcome, learning rate — lets the analyst ask precise diagnostic questions where ordinary language offers only "it isn't learning." The pattern also makes the baseline-shift phenomenon legible: as a predictor improves, errors shrink and learning slows, so shrinking error is a sign of approaching the ceiling rather than of failing effort. The clarifying force is to convert vague claims about feedback and improvement into a structured account: identify the predictor, identify the signal channel, compute the signed error, and read learning as a function of that error scaled by the learning rate — with zero error meaning either mastery or a sealed-off model, distinguishable only by perturbation.
Manages Complexity¶
The pattern collapses a family of cross-substrate phenomena — dopaminergic phasic bursts, TD errors, Kalman innovations, earnings surprises, disappointment, weather-forecast residuals, gradient updates — into one structural quantity: signal minus prediction, used as a teaching signal. A practitioner who understands this in one domain understands it in all of them, because the load-bearing object is identical and only the substrate machinery differs.
It also collapses a family of failure modes into one diagnostic question: which component is broken — predictor, signal channel, error computation, or update rule? The common pathologies sort cleanly: a persistently biased predictor (systematic error the update does not correct), a noisy or delayed signal (the error dominated by noise, so the learning rate must drop), an error computed against the wrong predictor (learning from someone else's surprise or a stale prediction), a learning rate too high (overshooting) or too low (frozen), and a censored outcome — losses never seen produce no error and no learning, the silent-failure mode at the heart of survivorship bias. By reducing a heterogeneous space of "why isn't this learning" stories to a four-component checklist, the pattern makes learning systems across substrates diagnosable with one shared vocabulary.
Abstract Reasoning¶
The pattern licenses several characteristic moves. No surprise, no learning: a system whose outcomes always match predictions has stopped updating regardless of whether the predictions are correct, so persistent zero error can mean mastery or sealed-off delusion, distinguishable only by perturbation. Probe with the unexpected: to elicit learning, deliberately introduce stimuli the predictor has not modelled — adversarial examples, exploration bonuses, curriculum jumps all exploit this.
Calibrate the learning rate to noise: small rates protect against overfitting individual errors in noisy environments, large rates speed convergence in stable ones, and the optimum is the Kalman-gain logic. Diagnose by the residual: when learning stalls or behaves pathologically, examine the residual time series — systematic bias indicates the predictor's structure is wrong, growing variance indicates regime change, cyclical residuals indicate a missing predictor input. Block the channel, kill the learning: ablating the error signal — a dopamine lesion, a missing earnings report, censored loss data — freezes the predictor even as outcomes continue. And sign matters: positive and negative errors update different pathways in many real systems (D1 versus D2 dopamine receptors, loss aversion, appetitive versus aversive learning), so a monolithic "error magnitude" misses the asymmetry. The reasoner asks, at every turn: what is the predictor, what is the signed error against it, what is the learning rate relative to the noise, and what does the residual structure say is wrong?
Knowledge Transfer¶
Reward prediction error has one of the cleanest formal-equivalence stories in the catalogue: the same scalar — signal minus prediction, scaled by a learning rate — drives learning across dopaminergic neurons, TD-learning algorithms, Kalman filters, exponential-smoothing forecasters, gradient-descent learners, earnings-surprise asset pricing, and predictive-coding sensory hierarchies. The role mapping is exact: the predictor maps to the value function, the filter's state estimate, the consensus forecast, the cortical generative model; the signal maps to the reward, the observation, the actual earnings, the sensory input; the error maps to the TD error, the innovation, the earnings surprise, the bottom-up residual; and the learning rate maps to the step size, the Kalman gain, the price-impact coefficient, the plasticity rate.
The transfers are documented and quantitative. The formal equivalence between phasic dopamine responses and TD errors imported an algorithmic theory into biology and motivated whole-brain reward models. TD-learning models of consumer choice and asset pricing import error-driven updating directly into economics, with the expected-versus-unexpected decomposition as the structural commitment. Framing perception as the suppression of prediction error reorganises classical sensory hierarchies — top-down signals become predictions, bottom-up signals become residuals — and predicts effects (mismatch negativity, repetition suppression) that monolithic-feedforward models do not. Kalman innovations and TD errors being the same structural quantity lets robotics treat localisation and reward learning as instances of one update rule. The zone-of-proximal-development and desirable-difficulty literatures are the prediction-error principle applied to pedagogy — too-easy is no error, too-hard is uninformative error, optimal is calibrated surprise. And portfolio-manager and meteorologist habits of decomposing forecasts into expected-versus-surprise components port to any domain with calibrated predictions, because the surprise component is where the information lives. Two refinements travel with the prime and must be preserved: the baseline-shift phenomenon (shrinking error signals an approaching ceiling, not failing effort) and the positive/negative asymmetry (the two signs run through different pathways with different gains, which a homogeneous abstraction misses). The unifying transfer move is always: identify the predictor and the signal, compute the signed error between them, scale it by a learning rate matched to the noise, and read both learning and its absence off that error.
Examples¶
Formal/abstract¶
The Kalman filter's innovation term is reward prediction error stated as an exact recursive estimator, and it makes the learning-rate-matched-to-noise principle a derived optimum rather than a heuristic. The predictor is the filter's internal state estimate \(\hat{x}_{t|t-1}\) — its model of where a tracked quantity (a spacecraft's position, a sensor-fused robot pose) is expected to be. The prediction on a given step is the predicted observation \(H\hat{x}_{t|t-1}\): what the sensor should read if the estimate is right. The observed signal is the actual measurement \(z_t\). The signed scalar error — the filter's innovation — is exactly \(\nu_t = z_t - H\hat{x}_{t|t-1}\), signal minus prediction, and it is the only term that updates the estimate: \(\hat{x}_{t|t} = \hat{x}_{t|t-1} + K_t \nu_t\). The learning rate is the Kalman gain \(K_t\), and the prime's "calibrate the learning rate to noise" inference is here a theorem: \(K_t\) is computed from the ratio of the estimate's uncertainty to the measurement noise, so a noisy sensor automatically yields a small gain (trust the prediction, learn slowly) and a precise sensor a large gain (trust the measurement, learn fast). The prime's baseline-shift dynamic is visible in the covariance: as the filter converges, its uncertainty shrinks, the gain drops, and each innovation moves the estimate less — shrinking updates signal an approaching estimation ceiling, not failure. The diagnostic the prime licenses is the residual analysis practitioners actually use: if the innovation sequence shows systematic bias, the filter's model (the matrix \(H\) or the dynamics) is wrong; if its variance grows, a regime change has occurred; a well-tuned filter produces zero-mean white innovations. The intervention is the prime's four-component checklist applied to a filter that diverges — is the predictor's model wrong, the signal channel corrupted, the innovation mis-computed, or the gain mistuned?
Mapped back: The Kalman innovation is reward prediction error in exact form — state estimate as predictor, predicted observation as prediction, measurement as signal, innovation as the signed error, and Kalman gain as the noise-matched learning rate — and the optimality of gain-from-noise is the prime's learning-rate principle proven rather than asserted.
Applied/industry¶
Two domains far from filtering — earnings-surprise asset pricing in finance and desirable-difficulty design in education — run the same signal-minus-prediction structure. In equity markets, the predictor is the analyst consensus forecast of a company's earnings; the prediction is the consensus number going into the announcement; the observed signal is the actual reported earnings. The prime's load-bearing claim is the empirical core of the field: prices move on the earnings surprise — actual minus consensus — not on the earnings level. A company can report record profits and see its stock fall if the profits missed the consensus (a negative error), while a modest profit that beat expectations rises; fully anticipated earnings move nothing, because the expected is silent. The prime's sign-asymmetry refinement is observed directly: negative surprises often move prices more sharply than equal-sized positive ones (a loss-aversion asymmetry), so a homogeneous "surprise magnitude" misses real structure. The prime's censored-outcome failure mode is the survivorship-bias trap that quants guard against: losses that are never recorded (delisted firms dropped from the dataset) produce no error and no learning, silently biasing the model. Desirable-difficulty pedagogy maps cleanly: the learner is the predictor, each practice problem's outcome is the signal, and the prime's "no surprise, no learning" inference is the design principle — a problem the student already knows generates zero error and teaches nothing, a problem far beyond them generates an error too large to be informative, and the productive zone (the "zone of proximal development") is calibrated surprise where the signed error is large enough to teach but small enough to assimilate. The intervention the prime names is the curriculum designer's: tune the difficulty so the predicted-versus-actual gap stays in the informative band, and read a student's shrinking errors on a topic as approaching mastery (baseline shift) rather than as a need for ever-harder drilling.
Mapped back: Earnings-surprise pricing and desirable-difficulty learning both instantiate a predictor (consensus forecast; student model), a signal (actual earnings; problem outcome), and a signed error that drives the update (surprise moves prices; calibrated surprise drives learning), with the prime's sign-asymmetry and baseline-shift refinements visible in both, so the principle — read change off the error, not the outcome — transfers from finance to education unchanged.
Structural Tensions¶
T1 — Surprise versus Outcome Magnitude (sign/direction). Learning is driven by the signed gap between expected and received, not by the outcome's raw size; a large but fully predicted outcome teaches nothing. The failure mode is "more outcome, more learning" reasoning — escalating reward or stimulus that the predictor already anticipates and getting no update. Diagnostic: ask whether the outcome was expected. If the predictor already forecast it, the error is zero regardless of the outcome's magnitude; the lever to drive learning is to increase surprise (make the outcome less predictable), not to increase the outcome itself.
T2 — Zero Error as Mastery versus Sealed Model (measurement). Persistent zero error is ambiguous: it can mean the predictor is correct (mastery) or that the model has stopped receiving informative input (sealed-off delusion). The failure mode is reading vanishing error as success when the system has actually gone blind. Diagnostic: perturb the predictor with a novel input and watch whether an error appears. If a deliberately unexpected probe still produces zero error, the model is sealed rather than accurate; only perturbation distinguishes a correct predictor from one that no longer updates.
T3 — Learning Rate versus Noise (scalar). The learning rate must be matched to the signal's noise — too high overfits individual noisy errors, too low freezes learning in a stable environment. The two failure directions are opposite and both live. The failure mode is a fixed rate applied across changing noise regimes, overreacting to noise or underreacting to real change. Diagnostic: compare the update step to the signal's noise level. If the predictor lurches on single noisy observations, the rate is too high for the noise; if it ignores a genuine regime shift, too low — the Kalman-gain logic sets the rate from the uncertainty-to-noise ratio, not by convenience.
T4 — Baseline Shift versus Failed Effort (temporal). As a predictor improves, errors shrink and learning slows of its own accord; diminishing error can signal an approaching ceiling rather than failing effort. The failure mode is misreading the natural decay of error as a problem and escalating input — ever-harder drilling on a near-mastered topic. Diagnostic: ask whether shrinking error reflects mastery or a broken channel. If the predictor is demonstrably accurate and errors are small because little surprise remains, the system has reached its current ceiling; pushing harder wastes effort, whereas a sealed channel would show small error despite inaccuracy.
T5 — Positive versus Negative Error Asymmetry (sign/direction). Positive and negative errors often run through different pathways with different gains, so a monolithic "error magnitude" misses real structure. The failure mode is treating gains and shortfalls symmetrically when the system weights them differently — loss aversion, distinct appetitive/aversive learning, asymmetric price reactions to good versus bad surprises. Diagnostic: ask whether positive and negative errors of equal size produce equal updates. If under- and over-shoots are handled by separate mechanisms with different sensitivities, modeling the error as a single signed scalar will mispredict; the two signs must be tracked separately.
T6 — Observed Error versus Censored Outcome (scopal). The teaching signal only exists for outcomes the system actually observes; censored outcomes — losses never seen, failures filtered out — produce no error and no learning. The failure mode is survivorship bias: the predictor learns confidently from a sample that systematically excludes the informative failures. Diagnostic: ask whether the outcomes that would generate corrective errors are present in the channel. If failures are removed before they reach the error computation (delisted firms, silent attrition), the predictor is trained on a biased survivor set; the absence of error reflects a censored channel, not an accurate model.
Structural–Framed Character¶
Reward prediction error sits at the structural end of the structural–framed spectrum. Although it originates in computational neuroscience, the pattern it names — learning from the signed gap between predicted and received outcome, using that error as the teaching signal — is pure relational/computational structure, and on every diagnostic it reads structural, matching the frontmatter's all-zero criteria and aggregate of 0.0.
Walking the five diagnostics with this prime's substrates: vocabulary travels freely. The same predictor / signal / signed-error / learning-rate structure is told in phasic dopamine bursts in neuroscience, in the temporal-difference error in reinforcement learning, in the innovation term in Kalman filtering, in earnings surprise in finance, and in calibrated difficulty in education — each substrate names the error in its own words ("TD error," "innovation," "surprise," "residual"), importing no neuroscience lexicon; the scalar signal-minus-prediction is the same object everywhere, with documented formal equivalences. Evaluative weight is absent: a prediction error is neither good nor bad; a positive error is not "success" and a negative error not "failure," only signed information about surprise. Institutional origin is formal — the structure is fully stated as a signed scalar (signal minus prediction) scaled by a learning rate, with no appeal to human institutions. It is not human-practice-bound: it runs indifferently in midbrain dopamine circuits, in a Kalman filter tracking a spacecraft, and in a gradient-descent learner, none mediated by any human practice. And invoking it recognizes a pattern already present rather than importing a frame — to identify a reward prediction error is to point at a real predictor, a real signal, and the computable gap between them one can test by perturbing the predictor, not to overlay an interpretation. Every diagnostic points the same way, and the prime is structural without qualification.
Substrate Independence¶
Reward Prediction Error is a strongly substrate-independent prime — composite 4 / 5 on the substrate-independence scale. Its signature — a scalar discrepancy computed as outcome minus prediction, which is then used to update the prediction — is a clean relational object that travels cleanly across distinct domains: neuroscience (phasic dopamine encoding the error), reinforcement learning (the temporal-difference error that drives value updates), Bayesian filtering (the innovation term that corrects the estimate), finance (earnings surprise moving prices), and education (feedback as the gap between expectation and result). That spread gives it solid domain breadth, and its structural abstraction is high because the signal-minus-prediction skeleton carries no commitment to any medium. What holds the breadth and abstraction sub-scores at 4 rather than 5 is a residual lean toward learning-and-valuation settings that presuppose a predictor maintaining an expectation — the pattern needs an agent or estimator with a forecast to be corrected. The transfer evidence, by contrast, is rated 5: the temporal-difference learning formalism is the same object in machine learning and in the dopamine literature, an unusually well-documented, formally-carried cross-domain identity rather than an analogy. Strong but predictor-bound breadth with exemplary transfer places the composite at 4.
- Composite substrate independence — 4 / 5
- Domain breadth — 4 / 5
- Structural abstraction — 4 / 5
- Transfer evidence — 5 / 5
Relationships to Other Primes¶
Parents (2) — more general patterns this builds on
-
Reward Prediction Error is part of, typical Predictive Coding
RPE is the scalar teaching signal (signal minus prediction) that predictive_coding's hierarchy transmits — the error any such system uses. Part-of/component, NOT identity: predictive_coding is one system built FROM prediction errors. The file: 'predictive coding is one system built from prediction errors; RPE is the error.' Owner: this could also read as RPE being more general than predictive_coding — drawn as the file frames it (PC=architecture, RPE=its currency).
-
Reward Prediction Error is part of, typical Reinforcement
RPE is the prediction-error decomposition INSIDE reinforcement — it refines what reinforcement's 'value signal' actually is (surprise, not magnitude). A component of the reinforcement loop, not the whole selection engine.
Path to root: Reward Prediction Error → Predictive Coding → Feedback
Neighborhood in Abstraction Space¶
Reward Prediction Error sits in a sparse region of abstraction space (81st percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely rather than landing on a neighbor.
Family — Anticipation & Forward Models (15 primes)
Nearest neighbors
- Reinforcement — 0.73
- Shortcut Learning — 0.70
- Observational Learning (Social Learning) — 0.68
- Concept Drift — 0.68
- Self-Defeating Prediction — 0.68
Computed from structural-signature embeddings · 2026-06-14
Not to Be Confused With¶
Reward prediction error's nearest neighbor is predictive_coding, and the two are tightly bound because predictive coding is built from prediction errors — but they sit at different levels. Predictive coding is an architecture: a hierarchical system in which each level sends predictions downward to the level below and the residual mismatch upward, so that perception (or cognition) becomes the progressive suppression of prediction error across the hierarchy. Reward prediction error is the scalar teaching signal itself — signal minus prediction, scaled by a learning rate — that any error-driven learner uses, whether or not it is organized hierarchically. The relationship is part-to-whole: predictive coding is one influential system whose currency is prediction errors, while RPE is the bare error that also drives a single-level TD learner, a Kalman filter, or an earnings-surprise model with no hierarchy at all. Conflating them leads to assuming that wherever there is a prediction error there must be a downward-prediction/upward-residual hierarchy (over-attributing architecture), or that predictive coding is "just" the RPE signal (missing the multi-level message-passing structure that gives predictive coding its distinctive predictions, like repetition suppression and mismatch negativity).
Reward prediction error must also be distinguished from reinforcement, with which it is intimately linked because RPE is what drives reinforcement in modern accounts. Reinforcement is the engine of selection: an action's consequence changes the action's future probability, with a repertoire of variants, a contingency, a schedule, and a differential update. Reward prediction error is the specific insight about what the consequence signal actually is: not the raw reward but the signed gap between received and expected reward, so a fully predicted reward (zero error) reinforces nothing while an unexpected one drives strong learning. RPE is thus the prediction-error decomposition operating inside the reinforcement loop — it refines reinforcement's "value signal" component, specifying that the value that updates behavior is surprise, not magnitude. The two are not interchangeable: reinforcement names the whole variation-contingency-schedule selection structure, while RPE names the error currency that the schedule's value signal turns out to be. A practitioner who collapses them might design a reinforcement schedule that escalates already-anticipated rewards (chasing magnitude) and be puzzled by the absent learning — exactly the error RPE's "surprise teaches, the expected is silent" corrects.
A third genuine confusion is with bayesian_updating, because both revise a model in light of new evidence and the Kalman filter is a shared example. The distinction is what gets updated and how much information the update carries. Bayesian updating revises an entire probability distribution — a posterior over hypotheses — by multiplying a prior by a likelihood, carrying full uncertainty information. Reward prediction error is a scalar point-estimate correction: a single signed residual, scaled by a learning rate, nudging a point prediction toward the observed signal. The two coincide only in the special Gaussian-linear case, where Bayesian updating reduces exactly to the Kalman recursion and the innovation is the prediction error with the Kalman gain as the optimal learning rate. Outside that case they diverge sharply: Bayesian updating maintains and propagates a full posterior (and can represent multimodal or skewed beliefs), whereas RPE keeps no distribution, only a running point estimate and a residual. Conflating them leads to crediting an RPE learner with uncertainty quantification it does not possess, or to treating a full Bayesian posterior update as if it could be summarized by a single scalar error — losing exactly the distributional information that distinguishes principled uncertainty handling from point-estimate tracking.
These distinctions matter because each isolates a different level or capacity: predictive coding is the hierarchical architecture (where RPE is the scalar error it transmits), reinforcement is the selection engine (where RPE refines what its value signal is), and Bayesian updating is full-distribution revision (where RPE is the scalar point correction that coincides only in the Gaussian case). A practitioner who conflates them over-attributes hierarchy, chases reward magnitude instead of surprise, or imagines uncertainty quantification an RPE learner lacks. Holding reward prediction error as the specific predictor / signal / signed-scalar-error / learning-rate structure keeps the analyst asking its real questions — whose prediction, against what predictor, with what learning rate relative to the noise, and what does the residual structure say is broken?
Solution Archetypes¶
No catalogued solution archetypes reference this prime yet.