26 lines
3.0 KiB
Markdown
26 lines
3.0 KiB
Markdown
## Brief history of human judgmental forecasting
|
||
|
||
- 1947–1991: Cold war failures of the US intelligence community
|
||
- Legendary figure: [Sherman Kent](https://www.wikiwand.com/en/Sherman_Kent), who came up with [words of estimative probability](https://www.wikiwand.com/en/Words_of_estimative_probability) to solve ambiguity in [National Intelligence Estimates](https://www.econlib.org/archives/2017/08/tristans_tetloc.html)
|
||
- Meanwhile:
|
||
- *An Analyst’s Reflections on Forecasting: The Limits of Prediction*, Bobby W. (anonymous)
|
||
- *Psychology of Intelligence Analysis](), by Richards J. Heuer
|
||
- 1988-2010: [Prediction Markets](https://www.wikiwand.com/en/Prediction_market) gain wider recognition, but they mostly fail to gain a foothold. An early small scandal in which an early project appears to contain [markets on terrorist attacks](https://www.wikiwand.com/en/Policy_Analysis_Market) sours the idea, at least initially. [Iowa Electronic Markets](https://www.wikiwand.com/en/Iowa_Electronic_Markets) is allowed to exist by US regulators because of low overall betting sizes and its academic purpose.
|
||
- 2010-2015: The [Aggregative Contingent Estimation (ACE) Program](https://www.wikiwand.com/en/Aggregative_Contingent_Estimation_(ACE)_Program) is run by IARPA, as a belated response from the US intelligence community to not having been able to predict the 2001 9/11 attacks. [ The Good Judgment Project ](https://www.wikiwand.com/en/The_Good_Judgment_Project) wins the competition, and in 2015, Phil Tetlock publishes [Superforecasting](https://www.wikiwand.com/en/Superforecasting:_The_Art_and_Science_of_Prediction). The Good Judgment Project fails to cross the [valley of death](https://acquisitiontalk.com/2019/12/explaining-the-valley-of-death-in-defense-technology/)
|
||
- 2015-2021: Crypto prediction markets such as [Augur](https://www.wikiwand.com/en/Augur_(software)), [Polymarket](https://polymarket.com/) and others allow users to trade without the difficult to get approval of US regulators. [PredictIt](https://www.wikiwand.com/en/PredictIt), founded in 2014, and [Kalshi](https://kalshi.com/), founded in 2020, negotiate with regulators to get said approval. Meanwhile, platforms like [Metaculus](https://www.wikiwand.com/en/Metaculus), founded in 2015 have users compete for internet points, thought they also have occasional monetary rewards for winners of tournaments.
|
||
|
||
## Human biases and their standard mitigations
|
||
- Vague verbiage → Quantitative forecasts, words of estimative probability
|
||
- General overconfidence → Calibration training
|
||
- Hindsight bias → Keeping track of predictions
|
||
- Scope insensitivity → Consider different quantities for your prediction. Anchor on the base-rate.
|
||
- Anchoring bias → If you have to anchor on something, anchor on the base-rate
|
||
- Confirmation Bias → Move a little bit with each piece of infomration
|
||
|
||
Other forecasting habits:
|
||
- Betting
|
||
- Teaming using the delphi method
|
||
- Emotional detachment
|
||
- Dialectical bootstrapping: Make a forecast, write down reasoning, forget about it, come back again, then average the two
|
||
- Pre & post-mortems
|