Expert Political Judgment¶
Tetlock, P. E. (2005). Expert Political Judgment: How Good Is It? How Can We Know?. Princeton University Press.
Cited by¶
5 citations across 5 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Calibration
- Tetlock (2005) showed how the choice of reference frame shapes expert judgment in his long-running study of political forecasting accuracy.
This sourceTwo-decade study of expert political/economic forecasts showing systematic overconfidence and frequent underperformance versus simple baselines, with reasoning style (fox vs hedgehog) shaping forecast quality.
Supported in partVerified against the publisher's abstract
Tetlock's study evaluates expert political forecasts and finds thinking style (fox versus hedgehog) affects accuracy, not 'reference frame' as such.
“He goes on to analyze which styles of thinking are more successful in forecasting.”
- Tetlock (2005) showed how the choice of reference frame shapes expert judgment in his long-running study of political forecasting accuracy.
- Epistemic Humility
- Historically, overconfident predictions in military strategy (the Maginot Line, the assumption that Germany would never invade; the belief that Vietnam would be a quick victory), economic forecasting (the 2008 financial crisis blindsided confident risk models; analysts consistently overestimated growth trajectories), medical diagnosis (misdiagnosed conditions, overconfident prognoses leading to unnecessary treatments), and policy (regime-change interventions based on overconfident geopolitical models) have led to preventable failures and human suffering, a pattern Tetlock (2005) documented across two decades of expert political forecasts that systematically underperformed simple statistical baselines.
This sourceReports a multi-decade study of ~28,000 expert forecasts showing experts were systematically overconfident and often outperformed by simple statistical baselines. SUPPORTS marker 225 (two decades of expert political forecasts that underperformed simple statistical baselines).
Supported in partVerified against the work's full text
Supports the Tetlock clause: 20+ years of expert forecasts barely beat chance and simple extrapolation; the Maginot Line, Vietnam, 2008 crisis and misdiagnoses are not in this source.
“Tetlock systematically collected a vast number of individual forecasts about political and economic events, made by recognised experts over a period of more than 20 years. He showed that these forecasts were not very much better than making predictions by chance, and also that experts performed only slightly better than the average person who was casually informed about the subject in hand.”
- Historically, overconfident predictions in military strategy (the Maginot Line, the assumption that Germany would never invade; the belief that Vietnam would be a quick victory), economic forecasting (the 2008 financial crisis blindsided confident risk models; analysts consistently overestimated growth trajectories), medical diagnosis (misdiagnosed conditions, overconfident prognoses leading to unnecessary treatments), and policy (regime-change interventions based on overconfident geopolitical models) have led to preventable failures and human suffering, a pattern Tetlock (2005) documented across two decades of expert political forecasts that systematically underperformed simple statistical baselines.
- Foreseeing (Prediction)
- Foresight
- Foresight makes room for trend-breaking futures — black-swan-style high-impact, low-probability events; regime shifts; phase transitions — by treating them as legitimate scenarios within the plural set rather than as off-distribution noise, the structural worry Taleb (2007) and Tetlock (2005) make central to their critiques of single-point forecasting.
This sourceTwo-decade study of 284 experts and 82,361 forecasts showing experts were systematically overconfident and often performed worse than simple statistical baselines — canonical demonstration of overconfidence costs in single-point prediction. SUPPORTS the critique-of-single-point-forecasting claim (165). [Forecast count corrected from 'nearly 28,000' to 82,361 from 284 experts.]
Supported in partVerified against the work's full text
The Princeton page for Expert Political Judgment shows Tetlock comparing fox and hedgehog forecasting styles; it says nothing of trend-breaking events or a plural scenario set.
“is more successful in predicting the future than the hedgehog, who knows one big thing, toils devotedly within one tradition, and imposes formulaic solutions on ill-defined problems.”
- Foresight makes room for trend-breaking futures — black-swan-style high-impact, low-probability events; regime shifts; phase transitions — by treating them as legitimate scenarios within the plural set rather than as off-distribution noise, the structural worry Taleb (2007) and Tetlock (2005) make central to their critiques of single-point forecasting.
- Wild Cards
- The method generates a conditional claim: if this event occurred, the consequences would be severe enough to warrant investment in contingency planning; therefore, it merits watchlist status and preparation. The converse claim — that named wild cards are likely to occur — does not follow from wild-card analysis, a separation Tetlock (2005) makes explicit in distinguishing scenario-based preparation from probabilistic prediction.
This sourceReports a two-decade study of nearly 28,000 expert forecasts showing that political and economic experts were systematically overconfident and frequently performed worse than simple statistical baselines—canonical empirical demonstration of overconfidence costs in policy-relevant prediction.
- The method generates a conditional claim: if this event occurred, the consequences would be severe enough to warrant investment in contingency planning; therefore, it merits watchlist status and preparation. The converse claim — that named wild cards are likely to occur — does not follow from wild-card analysis, a separation Tetlock (2005) makes explicit in distinguishing scenario-based preparation from probabilistic prediction.
Verification¶
Does it exist? Not checked yet. This entry carries no identifier to resolve. It was extracted from the citation as written in the article, normalized, and deduplicated against the rest of the registry.
Does it back the claim? Read against the text for 3 of 5 citations: 3 supported in part. Each verdict is shown under its citation below, with what in the work backs the sentence.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Registry ID ref:e83efc23ab97 · see in the full table