personal/nunosempere.github.io
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Light editing to the forecasting newsletter

Browse Source
This commit is contained in:
Nuno Sempere 2020-05-30 18:59:29 +02:00
parent 90e89d03b5
commit d1fa52004f
2 changed files with 132 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
ea/ForecastingNewsletter/May2020.md
View File

@ -64,13 +64,13 @@ At first I thought that, cunningly, the results of the first round would be used
### Coronavirus Information Markets: [coronainformationmarkets.com](https://coronainformationmarkets.com/)
For those who want to put their money where their mouth is, this is a prediction market for coronavirus related information.
Making forecasts is tricky, so would-be-bettors might be better off pooling their forecasts together with a technical friend. As of the end of this month, the total trading volume of active markets sits at a $26k+ (upwards from 8k last month), and some questions have been resolved already.
Making forecasts is tricky, so would-be-bettors might be better off pooling their forecasts together with a technical friend. As of the end of this month, the total trading volume of active markets sits at $26k+ (upwards from $8k last month), and some questions have been resolved already.
### Foretold: [foretold.io](https://www.foretold.io/) & EpidemicForecasting (c.o.i)
Epidemic Forecasting gathered a team of forecasters, including some of which had earnt the right to call themselves superforecasters because of their good performance on the Good Judgement project. They found initial success providing analysis and forecasts to countries and regions which wouldn't otherwise have the capacity, and advising a vaccine company on where to locate trials with as many as 100,000 participants; now they're looking for a project manager to take over: see [here](https://www.lesswrong.com/posts/ecyYjptcE34qAT8Mm/job-ad-lead-an-ambitious-covid-19-forecasting-project) for the pitch and for some more information.
Epidemic Forecasting gathered a team of forecasters, including some of which had earnt the right to call themselves superforecasters because of their performance on the Good Judgement project. They found initial success providing analysis and forecasts to countries and regions which wouldn't otherwise have the capacity, and advising a vaccine company on where to locate trials with as many as 100,000 participants; now they're looking for a project manager to take over: see [here](https://www.lesswrong.com/posts/ecyYjptcE34qAT8Mm/job-ad-lead-an-ambitious-covid-19-forecasting-project) for the pitch and for some more information.
For their part, Foretold has added a distribution drawer (written by me in reasonML, inspired heavily by the open-source code of Evan Ward's [probability.dev](https://probability.dev/)) to their [Highly Speculative Estimates](https://www.highlyspeculativeestimates.com/drawer) utility, and continued their partnership with Epidemic Forecasting.
For their part, Foretold has added a distribution drawer (written by myself in reasonML, inspired heavily by the open-source code of Evan Ward's [probability.dev](https://probability.dev/)) to their [Highly Speculative Estimates](https://www.highlyspeculativeestimates.com/drawer) utility, and continued their partnership with Epidemic Forecasting.
### Metaculus: [metaculus.com](https://www.metaculus.com/)
@ -114,7 +114,7 @@ One can sign-up on [their webpage](https://www.cset-foretell.com/). CSET was pre
## In The News.
Articles a announcement in more or less traditional news media, ordered by relevance.
- [Locust-tracking application for the UN](https://www.research.noaa.gov/article/ArtMID/587/ArticleID/2620/NOAA-teams-with-the-United-Nations-to-create-locust-tracking-application) (see [here](https://www.washingtonpost.com/weather/2020/05/13/east-africa-locust-forecast-tool/) is a take by the Washington Post), using software originally intended to track the movements of air polution. NOAA also sounds like a valuable organization: "NOAA Research enables better forecasts, earlier warnings for natural disasters, and a greater understanding of the Earth. Our role is to provide unbiased science to better manage the environment, nationally, and globally."
- [Locust-tracking application for the UN](https://www.research.noaa.gov/article/ArtMID/587/ArticleID/2620/NOAA-teams-with-the-United-Nations-to-create-locust-tracking-application) (see [here](https://www.washingtonpost.com/weather/2020/05/13/east-africa-locust-forecast-tool/) for a take by the Washington Post), using software originally intended to track the movements of air polution. NOAA also sounds like a valuable organization: "NOAA Research enables better forecasts, earlier warnings for natural disasters, and a greater understanding of the Earth. Our role is to provide unbiased science to better manage the environment, nationally, and globally."
- [United Nations: World Economic Situation and Prospects as of mid-2020](https://www.un.org/development/desa/dpad/publication/world-economic-situation-and-prospects-as-of-mid-2020/). A recent report is out, which predicts a 3.2% contraction of the global economy. Between 34 and 160 million people are expected to fall below the extreme poverty line this year. Compare with [Fitch ratings](https://www.fitchratings.com/research/sovereigns/further-economic-forecast-cuts-global-recession-bottoming-out-26-05-2020), which foresee a 4.6% decline in global GDP.
- [Fox News](https://www.fox10phoenix.com/news/cdc-says-all-models-forecast-increase-in-covid-19-deaths-in-coming-weeks-exceeding-100k-by-june-1) and [Business Insider](https://www.businessinsider.com/cdc-forecasts-100000-coronavirus-deaths-by-june-1-2020-5?r=KINDLYSTOPTRACKINGUS) report about the CDC forecasting 100k deaths by June the 1st, differently.
- Some transcient content on 538 about [Biden vs past democratic nomines](https://fivethirtyeight.com/features/how-does-biden-stack-up-to-past-democratic-nominees/), about [Trump vs Biden polls](https://fivethirtyeight.com/features/you-can-pay-attention-to-those-trump-vs-biden-polls-but-be-cautious/) and about [the USA vicepresidential draft](https://fivethirtyeight.com/features/its-time-for-another-2020-vice-presidential-draft/), and an old [review of the impact of VP candidates in USA elections](http://baseballot.blogspot.com/2012/07/politically-veepstakes-isnt-worth.html) which seems to have aged well. 538 also brings us this overview of [models with unrealistic-yet-clearly-stated assumptions](https://projects.fivethirtyeight.com/covid-forecasts/.)
@ -129,8 +129,8 @@ Articles a announcement in more or less traditional news media, ordered by relev
## Grab Bag
Podcasts, blogposts, papers, tweets and other recent nontraditional media.
- Some interesting discussion about forecasting over at Twitter, in [David Manheim](https://twitter.com/davidmanheim)'s, [Philip Tetlock](https://twitter.com/PTetlock)'s accounts, some of which have been incorporated into this newsletter. [This twitter thread](https://twitter.com/lukeprog/status/1262492767869009920) contains some discussion about how Good Judgement Open, Metaculus and expert forecasters fare against each other. In particular, note the caveats by @LinchZhang: "For Survey 10, Metaculus said that question resolution was on 4pm ET Sunday, a lot of predictors (correctly) gauged that the data update on Sunday will be delayed and answered the letter rather than the spirit of the question (Metaculus ended up resolving it ambiguous)." [This thread](https://twitter.com/mlipsitch/status/1257857079756365824) by Marc Lipsitch has become popular, and I personally also enjoyed [these](https://twitter.com/LinchZhang/status/1262127601176334336) [two](https://twitter.com/LinchZhang/status/1261427045977874432) twitter threads by Linchuan Zhang, on forecasting mistakes.
- [SlateStarCodex](https://slatestarcodex.com/2020/04/29/predictions-for-2020/) brings us a hundred more predictions for 2020. Some analysis by Zvi Mowshowitz [here](https://www.lesswrong.com/posts/gSdZjyFSky3d34ySh/slatestarcodex-2020-predictions-buy-sell-hold) and by [Bucky](https://www.lesswrong.com/posts/orSNNCm77LiSEBovx/2020-predictions).
- Some interesting discussion about forecasting over at Twitter, in [David Manheim](https://twitter.com/davidmanheim)'s and [Philip Tetlock](https://twitter.com/PTetlock)'s accounts, some of which have been incorporated into this newsletter. [This twitter thread](https://twitter.com/lukeprog/status/1262492767869009920) contains some discussion about how Good Judgement Open, Metaculus and expert forecasters fare against each other, but note the caveats by @LinchZhang: "For Survey 10, Metaculus said that question resolution was on 4pm ET Sunday, a lot of predictors (correctly) gauged that the data update on Sunday will be delayed and answered the letter rather than the spirit of the question (Metaculus ended up resolving it ambiguous)." [This thread](https://twitter.com/mlipsitch/status/1257857079756365824) by Marc Lipsitch has become popular, and I personally also enjoyed [these](https://twitter.com/LinchZhang/status/1262127601176334336) [two](https://twitter.com/LinchZhang/status/1261427045977874432) twitter threads by Linchuan Zhang, on forecasting mistakes.
- [SlateStarCodex](https://slatestarcodex.com/2020/04/29/predictions-for-2020/) brings us a hundred more predictions for 2020. Some analysis by Zvi Mowshowitz [here](https://www.lesswrong.com/posts/gSdZjyFSky3d34ySh/slatestarcodex-2020-predictions-buy-sell-hold) and by user [Bucky](https://www.lesswrong.com/posts/orSNNCm77LiSEBovx/2020-predictions).
- [FLI Podcast: On Superforecasting with Robert de Neufville](https://futureoflife.org/2020/04/30/on-superforecasting-with-robert-de-neufville/). I would have liked to see a more intense drilling on some of the points. It still gives pointers to interesting stuff, though, chiefly [The NonProphets Podcast](https://nonprophetspod.wordpress.com/), which looks like it has some more in-depth stuff. Some quotes:
> So it’s not clear to me that our forecasts are necessarily affecting policy. Although it’s the kind of thing that gets written up in the news and who knows how much that affects people’s opinions, or they talk about it at Davos and maybe those people go back and they change what they’re doing.
@ -149,20 +149,19 @@ Podcasts, blogposts, papers, tweets and other recent nontraditional media.
- [The advantages and limitations of forecasting](https://rwer.wordpress.com/2020/05/12/the-advantages-and-limitations-of-forecasting/). A short and sweet blog post, with a couple of forecasting anecdotes and zingers.
## The Rocky Horror Picture Show.
I have found negative stereotypes to be useful as a mirror with which to reflect on my own mistakes.
I have found negative examples to be useful as a mirror with which to reflect on my own mistakes; they may also be useful for shaping social norms.
- [Kelsey Piper of Vox harshly criticizes the IHME model](https://www.vox.com/future-perfect/2020/5/2/21241261/coronavirus-modeling-us-deaths-ihme-pandemic). "Some of the factors that make the IHME model unreliable at predicting the virus may have gotten people to pay attention to it;" or "Other researchers found the true deaths were outside of the 95 percent confidence interval given by the model 70 percent of the time."
- The [Washington post](https://www.washingtonpost.com/outlook/2020/05/19/lets-check-donald-trumps-chances-getting-reelected/) offers a highly partisan view of Trump's chances of winning the election. The author, having already made a past prediction, and seeing as how other media outlets offer a conflicting perspective, rejects the information he's learnt, and instead can only come up with more reasons which confirm his initial position. Problem could be solved with a prediction market.
- [California politics pretends to be about recession forecasts](https://calmatters.org/economy/2020/05/newsom-economic-forecast-criticism-california-model-recession-budget/). Problem could be solved with a prediction market. See also: [Simulacra levels](https://www.lesswrong.com/posts/fEX7G2N7CtmZQ3eB5/simulacra-and-subjectivity?commentId=FgajiMrSpY9MxTS8b); the article is at least three levels removed from consideration about bare reality. Key quote, about a given forecasting model: "It’s just preposterously negative... How can you say that out loud without giggling?" See also some more prediction ping-pong, this time in New Jersey, [here](https://www.njspotlight.com/2020/05/fiscal-experts-project-nj-revenue-losses-wont-be-as-bad-as-murphys-team-forecast/). Problem could be solved with a prediction market.
- [What Is the Stock Market Even for Anymore?](https://www.nytimes.com/interactive/2020/05/26/magazine/stock-market-coronavirus-pandemic.html). A New York Times claims to have predicted that the market was going to fall (but can't prove it with, for example, a tweet, or a hash of a tweet), and nonetheless lost significant amounts of his own funds. ("The market dropped another 1,338 points the next day, and though my funds were tanking along with almost everyone else’s, I found some empty satisfaction, at least, in my prognosticating.") The rest of the article is about said reported being personally affronted with the market not falling further ("the stock market’s shocking resilience (at least so far) has looked an awful lot like indifference to the Covid-19 crisis and the economic calamity it has brought about. The optics, as they say, are terrible.")
- [Forecasting drug utilization and expenditure: ten years of experience in Stockholm](https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-020-05170-0). A normally pretty good forecasting model had the bad luck of not foreseeing a Black Swan, and sending a study to a journal just before a pandemic, so that it's being published now. They write: "According to the forecasts, the total pharmaceutical expenditure was estimated to increase between 2 and 8% annually. Our analyses showed that the accuracy of these forecasts varied over the years with a mean absolute error of 1.9 percentage points." They further conclude: "Based on the analyses of all forecasting reports produced since the model was established in Stockholm in the late 2000s, we demonstrated that it is feasible to forecast pharmaceutical expenditure with a reasonable accuracy." Presumably, this has increased further because of covid, sending the mean absolute error through the roof. If the author of this paper bites you, you become a Nassim Taleb.
- In this time of need, where global cooperation might prove to be immensely valuable, Italy has lessons to share about how to forecast the coronavirus. The article [Forecasting in the Time Of The Coronavirus](https://www.bancaditalia.it/media/notizie/2020/en_Previsioni_al_tempo_del_coronavirus_Locarno_Zizza.pdf), by the Central Bank of Italy, is only available in Italian. Mysteriously, the press release, however, is in [English](https://www.bancaditalia.it/media/notizia/forecasting-in-the-time-of-coronavirus/).
- Some films are so bad it's funny. [This article fills the same niche](https://www.moneyweb.co.za/investing/yes-it-is-possible-to-predict-the-market/) for forecasting. It has it all: Pythagorean laws of vibration, epicycles, an old and legendary master with mystical abilities, 90 year predictions which come true. Further, from the [Wikipedia entry](https://en.wikipedia.org/wiki/William_Delbert_Gann#Controversy): "He told me that his famous father could not support his family by trading but earned his living by writing and selling instructional courses."
- [Austin Health Official Recommends Cancelling All 2020 Large Events, Despite Unclear Forecasting](https://texasscorecard.com/local/austin-health-official-recommends-cancelling-all-2020-large-events-despite-unclear-forecasting/). Texan article does not consider the perspective that one might want to cancel large events precisely *because* of the forecasting uncertainty.
- [Auditor urges more oversight, better forecasting at the United State's Department of Transport](https://www.wral.com/coronavirus/auditor-urges-more-oversight-better-forecasting-at-dot/19106691/): "Instead of basing its spending plan on project-specific cost estimates, Wood said, the agency uses prior-year spending. That forecasting method doesn't account for cost increases or for years when there are more projects in the works." The budget of the organization is $5.9 billion. Problem could be solved with a prediction market.
## Long content
This section contains items which have recently come to my attention, but which I think might still be relevant not just this month, but throughout the years. I dislike a bias towards recency in my content, and content in this section may or may not have been published in the last month.
This section contains items which have recently come to my attention, but which I think might still be relevant not just this month, but throughout the years. Content in this section may not have been published in the last month.
- [Pan-African Heatwave Health Hazard Forecasting](http://www.walker.ac.uk/research/projects/pan-african-heatwave-health-hazard-forecasting/). "The main aim, is to raise the profile of heatwaves as a hazard on a global scale. Hopefully, the project will add evidence to this sparse research area. It could also provide the basis for a heat early warning system." The project looks to be in its early stages, yet nonetheless interesting.
- [How to evaluate 50% predictions](https://www.lesswrong.com/posts/DAc4iuy4D3EiNBt9B/how-to-evaluate-50-predictions). "I commonly hear (sometimes from very smart people) that 50% predictions are meaningless. I think that this is wrong."
@ -210,6 +209,7 @@ handicappers. Each judge was presented with a list of 88 variables culled from t
- "We figured that a comparison between two students along the same dimension should be easier, cognitively, than a 13 comparison between different dimensions, and this ease of use should lead to greater reliance on the common dimension. The data strongly confirmed this hypothesis. Dimensions were weighted more heavily when common than when they were unique attributes. Interrogation of the subjects after the experiment indicated that most did not wish to change their policies by giving more weight to common dimensions and they were unaware that they had done so."
- "The message in these experiments is that the amalgamation of different types of information and different types of values into an overall judgment is a difficult cognitive process. In our attempts to ease the strain of processing information, we often resort to judgmental strategies that do an injustice to the underlying values and policies that we’re trying implement."
- "A major problem that a decision maker faces in his attempt to be faithful to his policy is the fact that his insight into his own behavior may be inaccurate. He may not be aware of the fact that he is employing a different policy than he thinks he’s using. This problem is illustrated by a study that Dan Fleissner, Scott Bauman, and I did, in which 13 stockbrokers and five graduate students served as subjects. Each subject evaluated the potential capital appreciation of 64 securities. [...] A mathematical model was then constructed to predict each subject's judgments. One output from the model was an index of the relative importance of each of the eight information items in determining each subject’s judgments [...] Examination of Table 4 shows that the broker’s perceived weights did not relate closely to the weights derived from their actual judgments.
- I informally replicated this with n=7 (some of which superforecasters).
- As remedies they suggest to create a model by elliciting the expert, either by having the expert make a large number of judgements and distillating a model, or by asking the expert what they think the most important factors are. A third alternative suggested is computer assistance, so that the experiment participants become aware of which factors influence their judgment.
- [Immanuel Kant, on Betting](https://www.econlib.org/archives/2014/07/kant_on_betting.html)

123
ea/ForecastingNewsletter/templongnow.txt
View File

@ -119,3 +119,126 @@ https://pdfs.semanticscholar.org/7a79/28d5f133e4a274dcaec4d0a207daecde8068.pdf
https://nunosempere.github.io/ea/Forecasting/AccuracyVsConfidence.png
https://www.econlib.org/archives/2014/07/kant_on_betting.html
https://archive.org/
https://mailchi.mp/18fccca46f83/forecastingnewsletter
https://www.augur.net/
https://bravenewcoin.com/insights/augur-price-analysis-token-success-hinges-on-v2-release-in-june
https://www.predictit.org/
http://electionbettingodds.com/
https://nunosempere.github.io/ea/Forecasting/electoral_college_predictit.png
https://www.predictit.org/markets/detail/6238/Will-Benjamin-Netanyahu-be-prime-minister-of-Israel-on-Dec-31,-2020
https://www.predictit.org/markets/detail/6265/Will-Trump-meet-with-Kim-Jong-Un-in-2020
https://www.predictit.org/markets/detail/6237/Will-Nicol%C3%A1s-Maduro-be-president-of-Venezuela-on-Dec-31,-2020
https://www.predictit.org/markets/detail/6674/Will-Kim-Jong-Un-be-Supreme-Leader-of-North-Korea-on-Dec-31
https://www.predictit.org/markets/detail/6702/Will-a-federal-charge-against-Barack-Obama-be-confirmed-before-November-3
https://www.predictit.org/markets/detail/3731/Will-Trump-switch-parties-by-Election-Day-2020
https://www.predictit.org/markets/detail/4632/Will-Michelle-Obama-run-for-president-in-2020
https://www.predictit.org/markets/detail/4614/Will-Hillary-Clinton-run-for-president-in-2020
https://www.predictit.org/markets/detail/2902/Will-the-2020-Democratic-nominee-for-president-be-a-woman
https://www.predictit.org/markets/detail/3633/Who-will-win-the-2020-Democratic-presidential-nomination
https://electionbettingodds.com/
https://ftx.com/
https://www.replicationmarkets.com
https://coronainformationmarkets.com/
https://www.foretold.io/
https://www.lesswrong.com/posts/ecyYjptcE34qAT8Mm/job-ad-lead-an-ambitious-covid-19-forecasting-project
https://probability.dev/
https://www.highlyspeculativeestimates.com/drawer
https://www.metaculus.com/
https://pandemic.metaculus.com/questions/4093/the-salk-tournament-for-coronavirus-sars-cov-2-vaccine-rd/
https://pandemic.metaculus.com/questions/4161/el-paso-series-supporting-covid-19-response-planning-in-a-mid-sized-city/
https://pandemic.metaculus.com/questions/4166/the-lightning-round-tournament-comparing-metaculus-forecasters-to-infectious-disease-experts/
https://pandemic.metaculus.com/COVID-19/
https://en.wikipedia.org/wiki/Regular_expression
https://en.wikipedia.org/wiki/Aggregative_Contingent_Estimation_(ACE
https://www.gjopen.com/
https://www.gjopen.com/questions/1499-before-1-january-2021-will-the-people-s-liberation-army-pla-and-or-people-s-armed-police-pap-be-mobilized-in-hong-kong
https://www.gjopen.com/questions/1495-will-the-winner-of-the-popular-vote-in-the-2020-united-states-presidential-election-also-win-the-electoral-college
https://www.gjopen.com/questions/1498-will-benjamin-netanyahu-cease-to-be-the-prime-minister-of-israel-before-1-january-2021
https://www.gjopen.com/questions/1621-before-28-july-2020-will-saudi-arabia-announce-the-cancellation-or-suspension-of-the-hajj-pilgrimage-scheduled-for-28-july-2020-to-2-august-2020
https://www.gjopen.com/questions/1551-will-formal-negotiations-between-russia-and-the-united-states-on-an-extension-modification-or-replacement-for-the-new-start-treaty-begin-before-1-october-2020
https://goodjudgment.com/covidrecovery/
https://goodjudgment.io/covid-recovery/#1384
https://goodjudgment.io/covid-recovery/#1373
https://goodjudgment.io/covid-recovery/#1374
https://goodjudgment.io/covid-recovery/#1363
https://medium.com/conversations-with-tyler/philip-tetlock-tyler-cowen-forecasting-sociology-30401464b6d9
https://cset.georgetown.edu/
https://www.cset-foretell.com/
https://www.openphilanthropy.org/giving/grants/georgetown-university-center-security-and-emerging-technology
https://www.research.noaa.gov/article/ArtMID/587/ArticleID/2620/NOAA-teams-with-the-United-Nations-to-create-locust-tracking-application
https://www.washingtonpost.com/weather/2020/05/13/east-africa-locust-forecast-tool/
https://www.un.org/development/desa/dpad/publication/world-economic-situation-and-prospects-as-of-mid-2020/
https://www.fitchratings.com/research/sovereigns/further-economic-forecast-cuts-global-recession-bottoming-out-26-05-2020
https://www.fox10phoenix.com/news/cdc-says-all-models-forecast-increase-in-covid-19-deaths-in-coming-weeks-exceeding-100k-by-june-1
https://www.businessinsider.com/cdc-forecasts-100000-coronavirus-deaths-by-june-1-2020-5?r=KINDLYSTOPTRACKINGUS
https://fivethirtyeight.com/features/how-does-biden-stack-up-to-past-democratic-nominees/
https://fivethirtyeight.com/features/you-can-pay-attention-to-those-trump-vs-biden-polls-but-be-cautious/
https://fivethirtyeight.com/features/its-time-for-another-2020-vice-presidential-draft/
http://baseballot.blogspot.com/2012/07/politically-veepstakes-isnt-worth.html
https://projects.fivethirtyeight.com/covid-forecasts/.
https://hbr.org/2020/05/why-economic-forecasting-is-so-difficult-in-the-pandemic
https://www.ibm.com/products/planning-analytics
https://www.channelasia.tech/article/679887/ibm-adds-ai-fuelled-forecasting-planning-analytics-platform/
https://www.cio.com/article/3544611/ibm-adds-ai-fueled-forecasting-to-planning-analytics-platform.html
https://finance.yahoo.com/news/synovus-financial-corp-earnings-missed-152645825.html
https://finance.yahoo.com/news/wienerberger-ag-earnings-missed-analyst-070545629.html
https://news.yahoo.com/park-lawn-corporation-earnings-missed-120314826.html
https://www.dailymail.co.uk/sport/tennis/article-8183419/amp/Wimbledon-set-net-huge-100m-insurance-payout-tournament-cancelled.html
https://nypost.com/2020/05/02/the-post-ranks-the-top-10-faces-in-new-york-sports-today/
https://nypost.com/2020/05/03/pitfall-to-forecasting-top-10-faces-of-new-york-sports-right-now/
https://www.scpr.org/news/2020/04/30/92263/in-forecasting-hurricane-dorian-models-fell-short/
https://www.nhc.noaa.gov/data/tcr/AL052019_Dorian.pdf
https://phys.org/news/2020-05-storm-chasers-life-saving.html
https://www.forbes.com/sites/marshallshepherd/2020/05/22/misunderstanding-of-coronavirus-predictions-is-eerily-similar-to-weather-forecasting/#2f1288467f75
https://www.dairyfoods.com/articles/94319-noun%C3%B3s-creamery-uses-demand-forecasting-platform-to-improve-production-process
https://twitter.com/davidmanheim
https://twitter.com/PTetlock
https://twitter.com/lukeprog/status/1262492767869009920
https://twitter.com/mlipsitch/status/1257857079756365824
https://twitter.com/LinchZhang/status/1262127601176334336
https://twitter.com/LinchZhang/status/1261427045977874432
https://slatestarcodex.com/2020/04/29/predictions-for-2020/
https://www.lesswrong.com/posts/gSdZjyFSky3d34ySh/slatestarcodex-2020-predictions-buy-sell-hold
https://www.lesswrong.com/posts/orSNNCm77LiSEBovx/2020-predictions
https://futureoflife.org/2020/04/30/on-superforecasting-with-robert-de-neufville/
https://nonprophetspod.wordpress.com/
https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2018SW002109
https://www.tandfonline.com/doi/full/10.1080/16742834.2020.1717329
https://www.eurekalert.org/pub_releases/2020-05/ioap-ata051520.php
https://www.cato.org/publications/white-paper/12-new-immigration-ideas-21st-century
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2463244
https://pubmed.ncbi.nlm.nih.gov/14561121/
https://www.nature.com/articles/d41586-020-01282-z
https://constancecrozier.com/2020/04/16/forecasting-s-curves-is-hard/
https://www.mercurynews.com/2020/05/20/letter-forecasting-state-expenses-for-budget-is-always-a-best-guess/
https://arxiv.org/pdf/2005.03297.pdf
https://rwer.wordpress.com/2020/05/12/the-advantages-and-limitations-of-forecasting/
https://www.vox.com/future-perfect/2020/5/2/21241261/coronavirus-modeling-us-deaths-ihme-pandemic
https://www.washingtonpost.com/outlook/2020/05/19/lets-check-donald-trumps-chances-getting-reelected/
https://calmatters.org/economy/2020/05/newsom-economic-forecast-criticism-california-model-recession-budget/
https://www.lesswrong.com/posts/fEX7G2N7CtmZQ3eB5/simulacra-and-subjectivity?commentId=FgajiMrSpY9MxTS8b
https://www.njspotlight.com/2020/05/fiscal-experts-project-nj-revenue-losses-wont-be-as-bad-as-murphys-team-forecast/
https://www.nytimes.com/interactive/2020/05/26/magazine/stock-market-coronavirus-pandemic.html
https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-020-05170-0
https://www.bancaditalia.it/media/notizie/2020/en_Previsioni_al_tempo_del_coronavirus_Locarno_Zizza.pdf
https://www.bancaditalia.it/media/notizia/forecasting-in-the-time-of-coronavirus/
https://www.moneyweb.co.za/investing/yes-it-is-possible-to-predict-the-market/
https://en.wikipedia.org/wiki/William_Delbert_Gann#Controversy
https://texasscorecard.com/local/austin-health-official-recommends-cancelling-all-2020-large-events-despite-unclear-forecasting/
https://www.wral.com/coronavirus/auditor-urges-more-oversight-better-forecasting-at-dot/19106691/
http://www.walker.ac.uk/research/projects/pan-african-heatwave-health-hazard-forecasting/
https://www.lesswrong.com/posts/DAc4iuy4D3EiNBt9B/how-to-evaluate-50-predictions
https://blog.cerebralab.com/Named%20Distributions%20as%20Artifacts
https://link.springer.com/article/10.3758/s13423-015-0947-8
https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/psychology-of-intelligence-analysis/
https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/csi-studies/studies/vol-63-no-4/Limits-of-Prediction.html
https://www.lesswrong.com/posts/ybYBCK9D7MZCcdArB/how-to-measure-anything
https://public.wmo.int/en/our-mandate/water/no-one-is-surprised-by-a-flood
https://public.wmo.int/en/media/news/caribbean-workshop-impact-based-forecasting-and-risk-scenario-planning
https://danluu.com/p95-skill/
https://arxiv.org/abs/cond-mat/0410063
https://www.lesswrong.com/posts/QkX2bAkwG2EpGvNug/the-second-law-of-thermodynamics-and-engines-of-cognition
https://pdfs.semanticscholar.org/7a79/28d5f133e4a274dcaec4d0a207daecde8068.pdf
https://nunosempere.github.io/ea/Forecasting/AccuracyVsConfidence.png
https://www.econlib.org/archives/2014/07/kant_on_betting.html
https://archive.org/