Ensemble Weighting Table¶
Artifact — instantiates Aggregation Function Design and Weighting
A standing table that fixes which judgment sources are in the pool and what reliability, calibration, and diversity weight each one carries — before any combining happens.
An Ensemble Weighting Table is the durable artifact that answers, for a set of models, experts, or signals, who counts how much and why — and stops there. Its defining feature, the one that separates it from the mechanisms that pool judgments, is that it produces a weight per source and the rationale for it, not a combined estimate. Each row is a source; each weight reflects a stated basis — demonstrated reliability (track-record accuracy), calibration (whether the source's stated confidence matches its hit rate), and diversity (how much independent information it adds versus how correlated it is with sources already in the pool). The table weights sources by trustworthiness, whereas a scoring rubric weights criteria by importance; and it deliberately never runs the blend — the actual pooling is a separate act performed downstream.
Example¶
A regional forecasting office ingests several numerical weather models for its next-day temperature guidance: two global models, a high-resolution regional model, and a statistical baseline. It maintains an ensemble weighting table that is updated each season. Two entries stand out. First, the two global models are known to be built on shared assimilation data and often err together; treating them as two independent votes would double-count that shared error, so the table gives them a diversity discount — jointly they carry less than the sum of their individual weights. Second, the regional model has been sharp but slightly over-confident, so its calibration column flags it and its weight is trimmed accordingly, while the humble statistical baseline earns a small, stable weight for the rare days everything else agrees and is wrong.
The table is the design: a documented map from each source to its weight and the reason. What it never does is compute the forecast — handing the weighted sources to a pooling step, and propagating their disagreement into the forecast interval, is a downstream job the table only feeds.
How it works¶
- Enumerate and vet the sources. List every model, expert, or signal eligible for the pool; a source not in the table has no say, and a duplicate silently doubles a voice.
- Attach a weight and its basis. Give each source a weight tagged as reliability-based, calibration-based, or diversity-based — never a bare number whose rationale is lost.
- Discount correlated sources. Where sources share data or bias, cut their combined weight so shared error is not mistaken for independent agreement.[n1]
- Version and revisit. Keep the table as a living artifact so weights update as track records and correlations shift, with the change history preserved.
Tuning parameters¶
- Weight basis mix — how much of each weight comes from reliability, calibration, or diversity. Leaning on measured accuracy sharpens the pool but overfits to past conditions; leaning on diversity guards against correlated failure but can prop up weak sources.
- Correlation discount strength — how hard correlated sources are down-weighted. Aggressive discounting protects against false agreement but may starve the pool of genuinely confirming evidence.
- Weight floor — whether every source keeps a minimum nonzero weight. A floor preserves rare-but-right outliers; dropping it lets the table concentrate on the best sources.
- Refresh cadence — how often weights are re-estimated. Frequent updates track regime change but chase noise; infrequent ones are stable but go stale.
When it helps, and when it misleads¶
Its strength is that it makes the influence of each judgment source explicit and revisable before anything is combined: reliability, calibration, and — critically — correlation are stated on the page, so the pool cannot silently treat three copies of the same bias as three confirmations. It is the natural place to encode hard-won knowledge about which sources to trust and which move together.
Its central failure mode is correlated-source overconfidence: if the diversity column is neglected, sources that share a blind spot get counted as independent votes, and the eventual blend looks far more certain than the evidence warrants. Reliability weights also overfit — a source weighted heavily for last year's accuracy can be exactly wrong when conditions change — and estimated weights can be less robust than a plain equal split. The classic misuse is stacking many similar experts or models to manufacture the appearance of consensus. The guarding discipline is to weight for diversity as well as accuracy, keep the correlation structure visible, and treat the table as a hypothesis about trust that gets re-checked against outcomes rather than frozen.
How it implements the components¶
input_item_set— the table is the roster: it fixes exactly which sources are in the pool and forces the questions of exclusion, duplication, and correlation.weight_assignment_scheme— each source's weight, tagged by its reliability/calibration/diversity basis and rationale, distributes influence across sources.
It does not implement aggregation_rule — it assigns the weights but never combines the sources; the pooling of weighted estimates is the job of the rule-family mechanisms here (and, across the encyclopedia, of Ensemble Decision Aggregation). Nor does it weight criteria to score competing options — that is Weighted Scoring Rubric's use of weighting, applied to criterion value rather than source reliability.
Related¶
- Instantiates: Aggregation Function Design and Weighting — the table supplies the justified source weights the pattern's combining step depends on.
- Sibling mechanisms: Weighted Scoring Rubric · Weight-Sweep Sensitivity Table · Median, Trimmed-Mean, or Quantile Rule · Ranked-Choice or Approval Voting Rule · Dashboard Rollup Formula · Aggregation Bias Audit
Editorial Notes¶
Form Classification¶
Form family: Rule, Policy & Commitment
Rationale: The table imposes standing eligibility and weighting rules that determine which sources may enter an ensemble and how reliability, calibration, diversity, and correlation affect their voice.
Nearest alternative: Representation, Specification & Plan — The artifact records weights and rationales, but those entries govern every subsequent combination rather than merely describe the source pool.
Review outcome: Adjudicated after independent review; high confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Statistical combination of estimates cohered explicit source weights based on reliability, calibration, and dependence before aggregation.
Related originating lineages:
- Data Science & Analytics — Model ensembling supplies performance-based and diversity-aware weighting of predictors.
- Operations Research — Expert-judgment aggregation supplies structured weights for heterogeneous human or analytic sources.
Review resolution: The current reviewers agree that statistics_experimental_design is primary. For the reported differences (reported_ambiguity, alternate_origin_disagreement, encyclopedia_synthesis_disagreement), the evidence supports cross_disciplinary_synthesis, multi_domain, and data_science, operations_research; these choices preserve materially formative origins without conflating later domain reach.
Attribution caveat: The durable table generalizes weighting across models, experts, and signals.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
The table is deliberately upstream of the blend. Keeping "who counts how much" separate from "combine them now" lets the weighting logic be argued, versioned, and audited on its own — and lets the same table feed a mean, a robust pool, or a full uncertainty propagation without re-litigating the weights each time.
[n1] An ensemble's error decomposes into the average error of its members minus their diversity (the ambiguity decomposition): combining helps only to the extent members err differently. This is the formal reason a weighting table must discount correlated sources — averaging near-identical judgments reduces error far less than their count suggests. ↩