add notes after reading 538 forecast

README.md

@@ -71,23 +71,102 @@ What's happening here is that:
   - In a normal democracy, like in Spain, a protest party could amass some electors, and use them as bargaining chips to govern together with one of the other major parties. For instance, this is what happened with Ciudadanos in Spain. Perhaps third parties performing strongly could conceivably, create pressure to reform the US electoral system.
   - In the US, with the system as currently exists, these votes seem to favour Trump.
 
+## Notes on other models
+
+**FiveThirtyEight** [2020](https://fivethirtyeight.com/features/how-fivethirtyeights-2020-presidential-forecast-works-and-whats-different-because-of-covid-19/), [2016](https://fivethirtyeight.com/features/a-users-guide-to-fivethirtyeights-2016-general-election-forecast/)
+
+Notes on 2020:
+
+- Adjusted for COVID pandemic
+  - Manually increased uncertainty
+- More fundamentals
+  - Looking back until 1880
+- Adjustments for changed partisanship
+- Covariance between states based on similarity metrics
+- Changes on how easy it is to vote
+- Polling averages. Explained further [here](https://fivethirtyeight.com/features/our-new-polling-averages-show-biden-leads-trump-by-9-points-nationally/)
+  - Polls as capturing a snapshot. Uncertainty should increase. Things can happen between now and the election.
+  - Weighted by pollster performance
+  - Trend line of the polls
+  - Likely voter adjustment
+  - Polling house adjustment
+  - "CANTOR" similarity scores
+  - "swinginess" of a state
+  - recency adjustments
+- Adjustements after major events. Debates, conventions, VP picks
+- Demographics, past voting patterns
+- Priors
+  - Incumbency
+  - Economic conditiosn
+- Partisan lean: in the last two elections
+  - In our partisan lean index, 75 percent of the weight is assigned to 2016 and 25 percent to 2012. So note, for example, that Ohio (which turned much redder between 2012 and 2016) is not necessarily expected to continue to become redder
+  - Home states of president and VP
+- Various complicated regressions
+  - One simple one is: polling for Northeast, Midwest, south, west
+- Ensemble forecast + polling average
+  - Weight depends on quantity of polling
+  - 55% to polling average in August
+  - 97% to well polled states towards the end of the campaign
+- Fundamentals based on economics
+- Index of economic conditions
+  - nonfarm payrolls
+  - spending
+  - income
+  - manufacturing
+  - inflation
+  - stock market
+  - normalized, weighted for recency
+- other factors
+  - incumbency
+  - polarization
+- forecast of those economic variables
+- relatively little weight to fundamentals, declining to zero by election day
+  - August: 77% to polling ensemble, 23% fundamentals
+- Accounting for uncertainty:
+  - national drift. 
+    Constant x (Days Until Election)^⅓ x Uncertainty Index
+  - national election day error. Errors in final polls since 1936.
+    - this is key, and tractable. [source](https://news.gallup.com/poll/110548/gallup-presidential-election-trial-heat-trends.aspx)
+    - More difficult to do this state by state, but it's a start
+    - Also doable in advance
+  - correlated state error
+    - also key
+    - based on demographics
+  - state-specific error
+- Uncertainty index. Its own involved thing.
+- 40,000 simulations each time the model is updated.
+  - This is relatively little, compared to my 10M
+- Not account for probability of faithless electors, nor shenanigans
+
+**[Gelman](https://projects.economist.com/us-2020-forecast/president/how-this-works)**
+
 ## Roadmap
 
 ### To do
 
-- [ ] Share with Samotsvety
+General:
+
 - [ ] Think about next steps
-- [ ] Consider conditional probabilities
+- [ ] Share with Samotsvety
-  - See how other models account for the correlation
-- [ ] Add uncertainty using Laplace's law of succession?
-  - Maybe only do this for contested states? Alabama is not going to turn Democratic?
-- [ ] Exclude partisan polls => not that many of them
 - [ ] Think about whether I want to monetize this
   - Maybe with Vox?
   - Otherwise: add MIT license & publish
 - [ ] Think about whether I want to add other collaborators
   - If so, add contribution sections, make available on github
 
+Steps to make this more accurate:
+
+- [ ] Think about correlation between states. 
+  - How?
+  - [ ] Consider conditional probabilities
+  - See how other models account for the correlation
+- [ ] Inspect polling stderrs
+- [ ] Add more years
+- [ ] Polling company errors
+- [ ] Make polling errors wider?
+- [ ] Economic fundamentals?
+- [ ] Print more data for polls
+
 ### Done 
 
 Incorporate base rates:
@@ -113,3 +192,9 @@ General
 - [x] Work on README
 - [x] Print states & polls separately
 - [x] Histogram distributions of electoral college votes
+
+### Discarded
+
+- [ ] ~~Add uncertainty using Laplace's law of succession?~~
+  - Maybe only do this for contested states? Alabama is not going to turn Democratic?
+- [ ] ~~Exclude partisan polls => not that many of them~~