3.0 KiB
3.0 KiB
Brief history of human judgmental forecasting
- 1947–1991: Cold war failures of the US intelligence community
- Legendary figure: Sherman Kent, who came up with words of estimative probability to solve ambiguity in National Intelligence Estimates
- Meanwhile:
- An Analyst’s Reflections on Forecasting: The Limits of Prediction, Bobby W. (anonymous)
- *Psychology of Intelligence Analysis](), by Richards J. Heuer
- 1988-2010: Prediction Markets gain wider recognition, but they mostly fail to gain a foothold. An early small scandal in which an early project appears to contain markets on terrorist attacks sours the idea, at least initially. Iowa Electronic Markets is allowed to exist by US regulators because of low overall betting sizes and its academic purpose.
- 2010-2015: The Aggregative Contingent Estimation (ACE) Program is run by IARPA, as a belated response from the US intelligence community to not having been able to predict the 2001 9/11 attacks. The Good Judgment Project wins the competition, and in 2015, Phil Tetlock publishes Superforecasting. The Good Judgment Project fails to cross the valley of death
- 2015-2021: Crypto prediction markets such as Augur, Polymarket and others allow users to trade without the difficult to get approval of US regulators. PredictIt, founded in 2014, and Kalshi, founded in 2020, negotiate with regulators to get said approval. Meanwhile, platforms like Metaculus, founded in 2015 have users compete for internet points, thought they also have occasional monetary rewards for winners of tournaments.
Human biases and their standard mitigations
- Vague verbiage → Quantitative forecasts, words of estimative probability
- General overconfidence → Calibration training
- Hindsight bias → Keeping track of predictions
- Scope insensitivity → Consider different quantities for your prediction. Anchor on the base-rate.
- Anchoring bias → If you have to anchor on something, anchor on the base-rate
- Confirmation Bias → Move a little bit with each piece of infomration
Other forecasting habits:
- Betting
- Teaming using the delphi method
- Emotional detachment
- Dialectical bootstrapping: Make a forecast, write down reasoning, forget about it, come back again, then average the two
- Pre & post-mortems