Thompson Sampling¶
A Bayesian bandit policy that samples a plausible reward model from the current posterior and chooses the action optimal under that sample, thereby probability-matching exploration to uncertainty.
Core Idea¶
Thompson sampling is a Bayesian policy for sequential decisions under uncertain rewards. At each round it samples one plausible reward model from the current posterior, then chooses the action with the highest expected reward under that sample; this makes action frequencies track the posterior chance of being optimal. Equivalently, each action is selected with the posterior probability that it is optimal. Equivalently, each action is selected with the posterior probability that it is optimal.
Scope of Application¶
The policy applies to repeated decisions with observable feedback and a probabilistic model linking contexts and actions to rewards. The policy applies to repeated choices where contexts, actions, rewards, and a posterior update can be specified.
- Multi-armed bandits. Uncertain arm reward rates are learned while rewards accumulate.
- Contextual recommendation. Posterior models condition expected reward on user or situation features.
- Online experiments. Traffic allocation shifts toward better variants while retaining uncertainty-driven exploration.
- Advertising. Creative or bid choices learn from sequential response.
- Approximate Bayesian control. Sampling methods make complex posterior models usable online.
Clarity¶
Specify the reward, horizon, action set, context, likelihood, prior, posterior-update rule, and sampling approximation. Distinguish Bayesian regret claims from frequentist guarantees and check whether feedback is delayed, censored, nonstationary, or action-dependent. Report randomized allocation when evaluating outcomes. The closest near miss sets the boundary: Upper-confidence-bound selection is the closest near miss: it explores actions with optimistic confidence bonuses rather than sampling a plausible model and acting optimally within it.
Manages Complexity¶
One posterior sample converts a difficult integration over all possible models into an ordinary optimization under one plausible world. Repetition automatically directs effort toward actions that are both promising and uncertain, coupling inference and control without a separate exploration schedule. The central exploration–exploitation tradeoff is this: Uncertain actions receive trials in proportion to their posterior chance of being best. A second model fidelity–online tractability tension matters because Rich reward models improve representation but make posterior sampling costly.
Abstract Reasoning¶
Use three linked moves: define a reward model and prior that reflect the action and context structure; update the posterior from observed context–action–reward tuples without treating chosen actions as exogenous evidence about the environment; draw a parameter or model realization from the current posterior. As a collapse test, the case exits when action randomization is not induced by posterior uncertainty about which action is optimal. A fourth check is to choose the action maximizing expected reward under that realization.
Knowledge Transfer¶
The posterior-sampling policy transfers from Bernoulli bandits to contextual, combinatorial, and some control settings when a coherent model and sample-optimal action can be computed. Randomized choice without posterior optimality is only analogy. No canonical parent prime is currently asserted; broader structural comparisons remain related-prime analogies until separately adjudicated in the DAG. A posterior draw realizes one plausible reward model.
Relationships to Other Abstractions¶
Current abstraction Thompson Sampling Domain-specific
Parents (1) — more general patterns this builds on
-
Thompson Sampling is a kind of Algorithm Prime
Thompson Sampling is a domain-specific kind of algorithm under its frozen identity and differentia.
Hierarchy paths (2) — routes to 2 parentless roots
- Thompson Sampling → Algorithm → Function (Mapping)
Neighborhood in Abstraction Space¶
Thompson Sampling sits in a moderately populated region (55th percentile for distinctiveness): it has near-neighbors but no dense thicket of look-alikes.
Family — Strategic Decision Biases & Mechanisms (29 abstractions)
Nearest neighbors
- MAP estimator — 0.87
- Bayesian Persuasion — 0.86
- Bayesian Programming — 0.86
- Particle Filter — 0.86
- M-Estimator — 0.84
Computed from structural-signature embeddings · 2026-10-08