personal/nunosempere.github.io
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update Self-experimentation-calibration.md

Browse Source
This commit is contained in:
Nuño Sempere 2019-05-06 16:08:51 +02:00 committed by GitHub
parent 106a5bb3a3
commit 49ffafddc6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 5 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
rat/Self-experimentation-calibration.md
View File

@ -32,11 +32,11 @@ My Brier score is 0.0755985. The significant digits become relevant later.
## 2. How do I compare to a some simple regression models?
I created four simple linear regression models and interpret their output as probability. I also consider a really really dumb predictor, for comparison purposes.
- 1. The first one regresses the binary outcome (1 if true, 0 if false) on the variables 3,4,7 and 10 outlined in the set up. That is, I consider the type of question, whether it was homework, normal or part of an exam, my score in the BDC and whether it was the first try or not.
- 2. The second includes all the data outlined in the set up, except my subjective probability. Or, in other words, all the above + {Hunch, Somewhat confident, Confident, Very confident, Incredibly Confident} as factors.
- 3. The third only regresses binary outcome on my subjective probability.
- 4. The fourth includes all the numerical factors outlined in the setup.
- 5. The fifth model is really dumb, and just outputs as a probability my total base rate. There were 505 questions, I got 451 right, so it outputs a probability of 451/505 every time. If it only gets partial data, it calculates the base rate for the data it has, and always predicts that afterwards.
1. The first one regresses the binary outcome (1 if true, 0 if false) on the variables 3,4,7 and 10 outlined in the set up. That is, I consider the type of question, whether it was homework, normal or part of an exam, my score in the BDC and whether it was the first try or not.
2. The second includes all the data outlined in the set up, except my subjective probability. Or, in other words, all the above + {Hunch, Somewhat confident, Confident, Very confident, Incredibly Confident} as factors.
3. The third only regresses binary outcome on my subjective probability.
4. The fourth includes all the numerical factors outlined in the setup.
5. The fifth model is really dumb, and just outputs as a probability my total base rate. There were 505 questions, I got 451 right, so it outputs a probability of 451/505 every time. If it only gets partial data, it calculates the base rate for the data it has, and always predicts that afterwards.
Here is a table: