feat: update blog

This commit is contained in:
Nuno Sempere 2022-09-01 10:11:24 +00:00
parent 15e357713f
commit fd04dd8967
19 changed files with 199 additions and 35 deletions

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.5 MiB

View File

@ -3,16 +3,14 @@ Introduction to Fermi estimates
*The following are my notes from an intro to [Fermi estimates](https://en.wikipedia.org/wiki/Fermi_problem) class I gave at [ESPR](https://espr-camp.org/), in preparation for a Fermithon, i.e., a Fermi estimates tournament.* *The following are my notes from an intro to [Fermi estimates](https://en.wikipedia.org/wiki/Fermi_problem) class I gave at [ESPR](https://espr-camp.org/), in preparation for a Fermithon, i.e., a Fermi estimates tournament.*
Fermi estimation is a method for arriving an estimate of an uncertain variable of interest. Given a variable of interest, sometimes you can decompose it into steps, and multiplying those steps together gives you a more accurate estimate than estimating the thing you want to know directly. I'll go through a proof sketch for this at the end of the post. Fermi estimation is a method for arriving an estimate of an uncertain variable of interest. Given a variable of interest, sometimes you can decompose it into steps, and multiplying those steps together gives you a more accurate estimate than estimating the thing you want to know directly. I'll go through a proof sketch for this at the end of the post.<p><img src="https://i.imgur.com/5uk7LBH.png" class="img-medium-center"></p>
If you want to take over the world, why should you care about this? Well, you may care about this if you hope that having better models of the world would lead you to make better decisions, and to better achieve your goals. And Fermi estimates are one way of training or showing off the skill of building models of the world. They have fast feedback loops, because you can in many cases then check the answer on the internet afterwards. But they are probably most useful in cases where you can't. If you want to take over the world, why should you care about this? Well, you may care about this if you hope that having better models of the world would lead you to make better decisions, and to better achieve your goals. And Fermi estimates are one way of training or showing off the skill of building models of the world. They have fast feedback loops, because you can in many cases then check the answer on the internet afterwards. But they are probably most useful in cases where you can't.
The rest of the class was a trial by fire: I presented some questions, students gave their own estimates, and then briefly discussed them. The rest of the class was a trial by fire: I presented some questions, students gave their own estimates, and then briefly discussed them. In case you want to give it a try before seeing the answers, the questions I considered were:
In case you want to give it a try before seeing the answers, the questions I considered were:
1. How many people have covid in the UK right now (2022-08-20)? 1. How many people have covid in the UK right now (2022-08-20)?
2. How many cummulative person years did people live in/under the Soviet Union? 2. How many cumulative person years did people live in/under the Soviet Union?
3. How many intelligent species does the universe hold outside of Earth? 3. How many intelligent species does the universe hold outside of Earth?
4. Are any staff members dating? 4. Are any staff members dating?
5. How many "state-based conflicts" are going on right now? ("state based conflict" = at least one party is a state, at least 25 deaths a year, massacres and genocides not included) 5. How many "state-based conflicts" are going on right now? ("state based conflict" = at least one party is a state, at least 25 deaths a year, massacres and genocides not included)
@ -23,7 +21,7 @@ In case you want to give it a try before seeing the answers, the questions I con
## 1. How many people have covid in the UK right now (2022-08-20)? ## 1. How many people have covid in the UK right now (2022-08-20)?
### My own answer ### My own [answer](https://www.squiggle-language.com/playground#code=eNp1jrsKwjAUhl8lZBTpzRsUHJ2kdChuhRDatD2Y5sSYtIj47qYFF7XLGf7L958nvXc4Fq7vuXnQ1Bon1rN0qsGi%2BSigwAKXxc1B20pRWAOqpSnVBjUaC6hyZ%2FMmjiJyJFEQE4tkV6rG8Goy2dgh6%2FggWIUD1D7zpxj6UyqN2kk%2Bl7Bhl7PP7g9BkmTeEqil%2BGUtrazIN2yJEYZ%2B5Do9vQm2GX29AQ8jZBI%3D)
``` ```
proportionOutOf100 = 0.1 to 5 proportionOutOf100 = 0.1 to 5
@ -33,13 +31,15 @@ people_who_have_covid = fraction_who_have_covid * population_of_UK
people_who_have_covid // 67k to 3.4M people_who_have_covid // 67k to 3.4M
``` ```
![](https://i.imgur.com/SNNpfge.png)
### Some student guesses ### Some student guesses
- 130k - 130k
- 600k - 600k
- 1M to 2M - 1M to 2M
### Check ### [Check](https://www.squiggle-language.com/playground#code=eNp9j0FPhEAMhf%2FKhJOaDQwYjdnEo6c9ciWZTJgCTaDFobgS43%2B3oBs163prXtvvvfeWTB0fy3kYfFySvcQZdpv0FFA4nhQkFPR9%2BTxj2%2FZQSkRqk31SMzUYBwgugJduciNEF%2FxiHk1R0Rh55CjI5Lg5XURYNQh6YtPCCJu8ImHx%2FdeFLi5iM%2FMvs6LaT%2BAarzSUxUUvoDjhK5vaPMut3anpNlybTGH%2BtecXDKbuPLWaSTqc0opwGHtUd4KjW4k%2Fe%2F2KmplzQ32nBuotYphV0UH%2F7taqhf2Gf4I2vK4vW96Yc97fFG10aw%2Brz%2F2DPWiP5P0Dbqavjg%3D%3D)
To check, we can use the number of confirmed deaths: To check, we can use the number of confirmed deaths:
@ -47,7 +47,7 @@ To check, we can use the number of confirmed deaths:
``` ```
confirmed_deaths_per_day = 2 confirmed_deaths_per_day = 2
proportion_of_deaths_reported = 0.2 to 1 proportion_of_deaths_reported = 0.2 t 1
total_deaths = confirmed_deaths_per_day / proportion_of_deaths_reported total_deaths = confirmed_deaths_per_day / proportion_of_deaths_reported
case_fatality_rate = to(0.01/100, 0.1/100) // paxlovid changed this. case_fatality_rate = to(0.01/100, 0.1/100) // paxlovid changed this.
implied_new_cases_per_day = total_deaths / case_fatality_rate implied_new_cases_per_day = total_deaths / case_fatality_rate
@ -56,16 +56,23 @@ implied_total_cases = implied_new_cases_per_day * infection_duration
implied_total_cases // 30K to 680K. implied_total_cases // 30K to 680K.
``` ```
## 2. How many cummulative person years did people live in/under the Soviet Union? ![](https://i.imgur.com/YK7vVKx.png)
### My own answer A [reader](https://www.reddit.com/r/TheMotte/comments/wy58g2/comment/ilw2vnn/?context=3) points out that the UK's Office for National Statistics [carries out randomized surveys](https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/bulletins/coronaviruscovid19infectionsurveypilot/26august2022) ([a](https://web.archive.org/web/20220826225309/https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/bulletins/coronaviruscovid19infectionsurveypilot/26august2022)). For the week of August 16th, they reported an estimated 1.3M cases across the UK, which falls within our initial estimate's interval, but not within our check's (?!).
## 2. How many cumulative person years did people live in/under the Soviet Union?
### My own [answer](https://www.squiggle-language.com/playground#code=eNp9jb0KwjAUhV%2FlklGUJlWRFlwKjp2yFkLQEC%2B0SU1vKkV8d2PRQUGnw%2FnhfDc2nP1Vxq7TYWIlhWiWc3Q4IfnwTtAhoW7lJaK1rZEU0FlWMj1a1fs%2BtprQOzX4EQ2p6JKBPWx5DeRhzXnduFc3GR2G1ImiELBKInaNOyb%2B%2FDEa9bX7R4AFfKyzDDbVk5jn1e9Tdn8ACl1ZaA%3D%3D)
``` ```
avg_population_soviet_union = 50M to 300M avg_population_soviet_union = 50M to 300M
soviet_years = 1991 - 1917 soviet_years = 1991 - 1917
cummulative_soviet_years = avg_population_soviet_union * soviet_years // 4B to 22B cumulative_soviet_years = avg_population_soviet_union * soviet_years
cumulative_soviet_years // 4B to 22B
``` ```
![](https://i.imgur.com/4Z5Argk.png)
Students guessed pretty much the same. Students guessed pretty much the same.
### Check ### Check
@ -94,23 +101,19 @@ The [dissolving the Fermi paradox](https://arxiv.org/abs/1806.02404) paper gives
<p><img src="./.images/dissolving-fermi.png" class="img-medium-center"></p> <p><img src="./.images/dissolving-fermi.png" class="img-medium-center"></p>
which gives something like a 40% chance of us being alone in the observable universe. I think the paper shows the importance of using distributions, rather than point-estimates: using point estimates results in the [Fermi paradox](https://en.wikipedia.org/wiki/Fermi_paradox). which gives something like a 40% chance of us being alone in the observable universe. I think the paper shows the importance of using distributions, rather than point-estimates: using point estimates results in the [Fermi paradox](https://en.wikipedia.org/wiki/Fermi_paradox). This is the reason why I've been multiplying distributions, rather than point estimates
The software I've been using for my own estimates is [Squiggle](https://www.squiggle-language.com/playground/); you can just copy examples there.
## 4. Are any staff members dating? ## 4. Are any staff members dating?
Note: Question does not include Junior Counsellors, because I don't know the answer to that. Note: Question does not include Junior Counselors, because I don't know the answer to that.
### Own answer ### Own answer
11 male and 3 women staff members. The camp has 11 male and 3 women staff members. So the number of heterosexual combinations is (11 choose 1) × (3 choose 1) = 11 × 3 = 33 possible pairings. However, some of the pairings are not compatible, because they repeat the same person, so the number of monogamous heterosexual pairings is lower.
So the number of combinations is (11 choose 1) × (3 choose 1) = 11 × 3 = 33 possible pairings? But some of the pairings are not compatible (repeat the same person)? Instead, say I'm giving a 1% to 3% a priori probability for any man-woman pairing. How did I arrive at this? Essentially, 0.1% feels too low and 5% too high. That implies a cumulative ~10% to 30% probability that a given woman staff member is dating any man staff member. Note that this rounds off nontraditional pairings—there is 1 nonbinary Junior Counsellor, but none amongst instructors, that I recall.
Instead, say a 1% to 3% a priori probability for any man-woman pairing? How did I arrive at this? Essentially, 0.1% feels too low and 5% too high. Implies a 10 to 30% probability that a given woman staff member is dating any man staff member. We round off nontraditional pairings (there is 1 nonbinary Junior Counsellor, but none amongst instructors, that I recall). If we run with the 30% chance for each woman staff member:
If 30% for woman staff member:
``` ```
p(0) = 1 × 0.7 × 0.7 × 0.7 = 34.3% p(0) = 1 × 0.7 × 0.7 × 0.7 = 34.3%
@ -147,23 +150,25 @@ You don't get to know the answer to this!
- At least 25 deaths a year. - At least 25 deaths a year.
- One sided violence, like massacres, are not included - One sided violence, like massacres, are not included
### My own estimate: ### My own [answer](https://www.squiggle-language.com/playground#code=eNqFjssKwjAQRX9lyEpLqdPagggu%2FYIuVUqtsQaSSU0TtIj%2FbrQ%2BsAvdzQz3zLkX1h70KXdKlaZjc2scDx%2Bn5U5YbV4XQcKKUuZHJ%2Bpa8twaQTWbM3KqqLQjv%2FMWFqDOowQRrIYpYgjPOUUcr6kxutHGCk0fpHDU2nIruWcxwuyexmi6pv4x7aWobFs03LyTT7jrbbGXPBQhrDCahZ5ONuMB76PfRQP4XSaA%2F%2F6hYzKBOL1XiTNk1xvgWnoq)
``` ```
num_countries = mx(200 to 300, 200 to 400) num_countries = mx(200 to 300, 200 to 400)
proportion_countries_unstable = 0.05 to 0.3 proportion_countries_unstable = 0.05 to 0.3
num_conflicts_per_unstable_country = mx(1, 2 to 4, [0.8, 0.2]) num_conflicts_per_unstable_country = mx(1, 2 to 4, [0.8, 0.2])
num_conflicts = num_countries * proportion_countries_unstable * num_conflicts_per_unstable_country // 14 to 150 num_conflicts = num_countries * proportion_countries_unstable * num_conflicts_per_unstable_country
num_conflicts // 14 to 150
``` ```
![](https://i.imgur.com/i5onQt7.png)
### Check ### Check
<iframe src="https://ourworldindata.org/grapher/the-number-of-active-state-based-conflicts?country=~OWID_WRL" loading="lazy" style="width: 100%; height: 600px; border: 0px none;"></iframe> <iframe src="https://ourworldindata.org/grapher/the-number-of-active-state-based-conflicts?country=~OWID_WRL" loading="lazy" style="width: 100%; height: 600px; border: 0px none;"></iframe>
## 6. How much does ESPR cost? ## 6. How much does ESPR cost?
### My own estimate ### My own [answer](https://www.squiggle-language.com/playground#code=eNp1jcEKwjAMhl%2Bl9KRDWCfKZLCjpx17LYziyix0yWxTZIjvbjsY6MFD%2BMMXvj8vHu74lHGatF94Qz6aw4qugyX0G7FgyWonH9GOozOSvIWRNxzi1M8GZ2dYy061ghsGSsTnCQjrOuglXSshGGEOoSB7CYfMzwoISbs%2Buwl8dRbsX1%2FBtoofuyxZLbr8p65S7o4X0e0V8PcHUqtRhA%3D%3D)
``` ```
num_people = 47 num_people = 47
@ -173,17 +178,19 @@ total_cost = num_people * cost_per_person_per_day * num_days
total_cost // 70K to 710K (280K) total_cost // 70K to 710K (280K)
``` ```
![](https://i.imgur.com/5PJq1ch.png)
### Student estimates ### Student estimates
Student estimates ranged from ~$50k to $150k Student estimates ranged from ~$50k to $150k
### Check ### Check
Per [this source](https://www.openphilanthropy.org/grants/european-summer-program-on-rationality-general-support/), ESPR cost $500k over two years, i.e., $250k per camp back when there was only one camp a year. Some of the students forgot about the cost of staff members, food, or common rooms. Per [this source](https://www.openphilanthropy.org/grants/european-summer-program-on-rationality-general-support/), ESPR cost $500k over two years. The money was used for a variety of events, including an online version in 2020 and a range of smaller events. If we count four events, the price tag per event is around $125k; if we only count two summer camps, it's up to $250k, but probably somewhere around $170k. Some of the students forgot about the cost of staff members, food, or common rooms.
## 7. How many people are members of the Chinese communist party ## 7. How many people are members of the Chinese communist party
### Own estimate ### My own [answer](https://www.squiggle-language.com/playground#code=eNqNjkELgkAUhP%2FKsseI3BAvQpegoyevwmK26QP3PX37lpDov6eCUJfoNsx8zMxTh44eZfS%2B5knnwtHtV%2BtyAyHeHEAQqPtyjNC2vSuFAVuda4zeNh2gC06d1PFc4cA0EAsQWrpvmQW0DXkfEYLYoWaZZtwcjMmU0CLSCteub8h656%2BOwwx%2FLu3Uvyu%2FW5NEZcVyIDVFhfr1Bj3QZNo%3D)
``` ```
num_chinese = 1B num_chinese = 1B
@ -192,13 +199,15 @@ num_communist_party_members = num_chinese * proportion_of_chinese_in_communist_p
num_communist_party_members // 5M to 30M num_communist_party_members // 5M to 30M
``` ```
![](https://i.imgur.com/qLRFz0q.png)
### Check ### Check
[Wikipedia](https://en.wikipedia.org/wiki/Chinese_Communist_Party#:~:text=As%20of%202022%2C%20the%20CCP,India's%20Bharatiya%20Janata%20Party.) reveals that there are 96M ~ 100M members, which around 10% of their population. Surprising. [Wikipedia](https://en.wikipedia.org/wiki/Chinese_Communist_Party#:~:text=As%20of%202022%2C%20the%20CCP,India's%20Bharatiya%20Janata%20Party.) reveals that there are 96M ~ 100M members, which around 10% of their population. Surprising.
## What is the US defense budget? ## What is the US defense budget?
### My own estimate ### My own [answer](https://www.squiggle-language.com/playground#code=eNp9jk%2BLwjAQxb%2FKkFOVYlNrl0XYi%2BDRUz0KodI0BtpMTBN2Rfa770Qq3T9lL3nkvXnzmzsbLvhehb6v3Y1tvQsyfVj7Rnt0T0cb7XXdVdeglepk5Z02im2ZaqzAVgy21gbeID%2BejEUbutprNN%2BTDT%2F8jsJAfsFjMK55OEm%2BKsEjFAtYwp%2FGEn4gM5jBEcihRTfVRCx5FI1spRkkUfqPhK94EUGkmzKNsh6%2F5eJkqDROi3NolPRUms6ky%2F5HzC3IMihyvouQ1xfSpKSXUOzzC9ZhiMw%3D)
``` ```
gdp_of_spain = 1T gdp_of_spain = 1T
@ -210,13 +219,15 @@ us_defense_budget = gdp_of_us * proportion_of_us_gdp_to_defense
us_defense_budget // 310B to 860B (560B) us_defense_budget // 310B to 860B (560B)
``` ```
![](https://i.imgur.com/y9SwK2R.png)
### Check ### Check
$715B in 2022, per [Wikipedia](https://en.wikipedia.org/wiki/Military_budget_of_the_United_States#Budget_for_FY2022) $715B in 2022, per [Wikipedia](https://en.wikipedia.org/wiki/Military_budget_of_the_United_States#Budget_for_FY2022)
## How many daily viewers does Tucker Carlson have? ## How many daily viewers does Tucker Carlson have?
### My own estimate ### My own [answer](https://www.squiggle-language.com/playground#code=eNp9jsEKgkAQhl9l2VNFpCYGCR07epJuwrLYqkO6a%2BNuJtG7t2lBWHYa%2BP9vZr4bbQrVxqaqOHY01GjEso%2F2R9AK3wlI0MDL%2BGwgz0sRawSZ05DWqjYl16AkUxk7xGRHfNeNEpkhT%2Fu45TotmBRtw1DkFsays5S7cgOilZ3%2BJ1yAhYeVTF17zh%2BwYALTJj0J7MnXwW0ih5BdQLQCmwJq23%2BpLhJJyH%2FPETKym26H%2F788HId4XkRm%2FmoTPXXXQTRPJL0%2FAIk%2BjW0%3D)
``` ```
population_of_US = 300M population_of_US = 300M
@ -230,20 +241,37 @@ tucker_viewership = population_of_US *
tucker_viewership // 11M (3.6M to 25M) tucker_viewership // 11M (3.6M to 25M)
``` ```
![](https://i.imgur.com/Ths8M1R.png)
### Check ### Check
Per [this source](<https://deadline.com/2022/08/fox-news-viewership-july-tucker-carlson-1235084361/>), Carlson gets about 3M viewers, so a bit outside my 90% range. I think I mis-estimated the fraction which watch news regularly, should probably have had a section for cable news specifically. Per [this source](<https://deadline.com/2022/08/fox-news-viewership-july-tucker-carlson-1235084361/>), Carlson gets about 3M viewers, so a bit outside my 90% range. I think I mis-estimated the fraction which watch news regularly, should probably have had a section for cable news specifically.
## Proof sketch that Fermi estimates go through ## Proof sketch that Fermi estimates produce better estimates than guessing directly
- Assume that guesses are independent lognormally distributed random variables, with a mean centered on the correct amount A [lognormal](https://en.wikipedia.org/wiki/Log-normal_distribution) distribution whose logarithm is a normal. It normally looks something like this:
- Observe that the multiplication of two lognormally distributed random variables is the sum of the logs of those variables, which are normally distributed.
- Per [well known math](https://en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables), the sum of two normally distributed variables is itself a normally distributed variable. And hence the product of two lognormally distributed variables is lognormally distributed. <p><img src="https://i.imgur.com/AjNygK1.png" class="img-medium-center"></p>
- Look at the shape of _standard deviation / mean_ of the underlying normals.
but if we display it with the axis in log scale—such that the distance between 30 and 40 is the same as the distance between 300 and 400, or between 3k and 4k, or such that the distance between 30 and 300 is the same as the distance between 300 and 3k—, it looks like a normal:
<p><img src="https://i.imgur.com/M7G5ZV2.png" class="img-medium-center"></p>
I think that a lognormal is a nice way to capture and represent the type of uncertainty that we have when we have uncertainty over several orders of magnitude. One hint might be that, per the [central limit theorem](https://en.wikipedia.org/wiki/Central_limit_theorem), sums of independent variables will tend to normals, which means that products will tend to be lognormals. And in practice I've also found that lognormals tend to often capture the shape of my uncertainty.
So then here is our proof sketch that you can get better estimates by decomposing a problem into steps:
- Assume that guesses are independent lognormals, with a mean centered on the correct amount. This traditionally means that your guesses are lognormally distributed random variables, that is, that you are drawing your guess from a lognormal. But you could also think of giving your guess as a distribution in the shape of a lognormal, as I have done in the examples above.
- Observe that the multiplication of two lognormals is the sum of the logs of those variables, which are normally distributed.
- Per [well known math](https://en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables), the sum of two normals is a normal. And hence the product of two lognormals is a lognormal.
- Look at the shape of _standard deviation / mean_ of the underlying normals.
- E.g., consider _n_ lognormals with mean _m_ and standard deviation _s_ on their underlying normals. - E.g., consider _n_ lognormals with mean _m_ and standard deviation _s_ on their underlying normals.
- Then that ratio is _s / m_ - Then that ratio is _s / m_
- Then consider the product of _n_ such lognormals - Then consider the product of _n_ such lognormals
- Per [this](https://stats.stackexchange.com/questions/212690/the-product-of-two-lognormal-random-variables), the mean of the underlying normal is going to be _n * m_, and the standard deviation of the underlying normal is going to be _sqrt(n)*s_ - Per [this](https://stats.stackexchange.com/questions/212690/the-product-of-two-lognormal-random-variables), the mean of the underlying normal of that product is going to be _n * m_, and the standard deviation of the underlying normal of that product is going to be _sqrt(n)*s_
- Then the ratio is going to be _sqrt(n) * s / (n * m)_, which converges to 0 as _n_ grows higher. - Then the ratio is going to be _sqrt(n) * s / (n * m)_, which converges to 0 as _n_ grows higher.
- We will get a similar phenomenon when we use lognormals that are not idential.
- Then, if you have the ability to make estimates with a certain degree of relative error—i.e., such that the _standard deviation / mean_ of your estimates is roughly constant—then you are better off decomposing your estimate in chunks and estimating those chunks.
I then have to argue that the ratio of _standard deviation / mean_ is a meaningful quantity, because the proof wouldn't go through if we had used _mean / variance_. But the _standard deviation / mean_ is known as the [coefficient of variation](https://en.wikipedia.org/wiki/Coefficient_of_variation), and it doesn't depend on the unit of measurement used—e.g,. centimeters vs meters—unlike some equivalent measure based on the variance.
One would then have to argue that the ratio of _standard deviation / mean_ is a meaningful quantity, because the proof wouldn't go through if we had used _mean / variance_. For what it's worth, _stardard deviation / mean_ is known as the [coefficient of variation](https://en.wikipedia.org/wiki/Coefficient_of_variation), and it doesn't depend on the unit of measurement used (e.g,. centimeters vs meters).

Binary file not shown.

After

Width:  |  Height:  |  Size: 116 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

View File

@ -0,0 +1,64 @@
A comment on Cox's theorem and probabilistic inductivism.
=====================================================================================
I'm currently reading _What is this thing called science_, an introduction to philosophy of science by Alan Chalmers. Around page 48, Chalmers presents the following chain of reasoning:
> One attempt to avoid the problem of induction involves weakening the demand that scientific knowledge be proven true, and resting content with the claim that scientific claims can be shown to be probably true in the light of the evidence. So the vast number of observations that can be invoked to support the claim that materials denser than air fall downwards on earth, although it does not permit us to prove the truth of the claim, does warrant the assertion that the claim is probably true.
> In line with this suggestion we can reformulate the principle of induction to read, if a large number of As have been observed under a wide variety of conditions, and if all these observed As have the property B, then all As probably have the property B. This reformulation does not overcome the problem of induction. The reformulated principle is still a universal statement. It implies, on the basis of a finite number of successes, that all applications of the principle will lead to general conclusions that are probably true.
> Consequently, attempts to justify the probabilistic version of the principle of induction by an appeal to experience involve an appeal to inductive arguments of the kind that are being justified just as the principle in its original form did.
I think this is not a good representation of what the probabilistic view would say. Here is what I think is a more convincing argument:
1. Per [Cox's theorem](https://en.wikipedia.org/wiki/Cox%27s_theorem), any "reasonable" view of probability is Bayesian.
2. Per Bayesian probability, hypotheses which predict future events which happen become more plausible, whereas hypotheses which don't predict the future loose plausibility.
3. Hypotheses such as "induction works", or "induction works in physics" perform well. Hypotheses such as "induction does't work", or "induction works well in the [soft sciences](https://en.wikipedia.org/wiki/Replication_crisis)" perform poorly.
4. As a result, we should view "induction works" as a highly probable statement.
To elaborate on this a bit, Cox's theorem starts with a few reasonable assumptions, like:
- "plausibility can be represented as a real number"
- "ways to derive plausibility have to be consistent"
- The plausibility of A & B is given by some function of "the plausibility of" and "plausibility of B given that A is the case", and this function is monotonic[^1].
- In the case of certainty, probability reverts to the logic of predicates.
- etc.
Using these assumptions, we arrive at:
<p><img src="https://i.imgur.com/gkymKm9.png"></p>
and by substiting P = w^m, we arrive at:
<p><img src="https://i.imgur.com/xw3pvyx.png" class="img-medium-center"></p>
For a good intro to Cox's theorem, see the first chapters in E.T. Jaynes' _Probability Theory: The Logic of Science_.
In any case, if C is our background knowledge, A is "induction works ok in physics", and B is "induction worked ok in this particular instance", then the probability of A rises as we observe more Bs. Conversely, the probability that induction doesn't work, or works badly, goes down every time that we observe that induction steps do work. A could also be "the sun will continue rising", and B could be "the sun has risen on day i".
To say this in other words: Every time we observe one induction step working ("if x has been the case for 6 steps, it will also be the case for 6+1 steps"), we increase our probability that "induction works for x generally", and we increase our probability that "induction works ok when we try to apply it practice".
So we get around the infinite regress in the problem of induction by bootstrapping the probability of "induction works" and "induction works in this particular case".
Now, in the spirit of philosophical questioning, we can point out a few problems and complications with this.
1. We can question the assumptions of Cox's theorem. One popular assumption to question is that the degree of plausibility ought to be represented by a real number.
2. We can point out that humans can only imperfectly apply Bayesian probability. In particular, it is very much non-trivial to deduce the implications of a theory about future events.
3. This scheme doesn't work if the initial, prior, degree of plausiblity that we assign to the principle of induction is zero or very (extremely) low. But because we have observed induction working ok so many times, starting with any reasonable probability (e.g,. 0.0001%, 0.1%, 1%, 10%) should lead us to increase our plausibility of induction to very high levels.
4. The principle of induction does in fact not work some times (e.g., when we encounter literal or metaphorical black swans, when we observe the replication crisis, or when we predict that the sun will go out in the far future). So the probabilistic view would not assign probability 100% to the principle of induction. And it would in fact assign different plausibilities to it depending on the domain. Teasing this out seems like it could be a bit tricky.
5. You sometimes have situations which are anti-inductive! E.g., as an authority becomes more reliable people trust it more, which means that it is more profitable or advantageous to subvert it, e.g., for political means.
6. Induction may not be a good guide in the presence of [anthropic effects](<https://en.wikipedia.org/wiki/Anthropic_Bias_(book)>).
7. There might be the suspicion that we have reduced the problem of induction to something which is ultimately equivalent.
8. We can think about . I get the impression that past philosophers were mostly considering whether induction works "as a whole", but if we are doing Bayesian probability, we have to assign probability mass to many other hypothesis and classes of hypotheses, like "induction works reliably in the hard sciences and not in the soft sciences".
Personally I am most worried about 2. and 5, and I view 4. as as a feature, not as a bug. But I also think that the scheme works _in practice_, and I'm very much attracted to Bayesian epistemology as a foundation for knowledge.
I don't think that 7. is the case: we could imagine a world in which past behavior is not correlated with future behaviour, and in which our probability machinery picks that up. For instance, in very liquid markets, it could be that the moment useful patterns are identified, they are exploited until they disappear, and we could easily update on that[^2].
I'm also excited about moving these problems away from philosophy and into [engineering](https://en.wikipedia.org/wiki/Information_theory) and practice, maybe using mechanisms such as [replication markets](https://replicationmarkets.com/) or [fraud markets](https://insightprediction.com/m/183/will-ryan-enos-have-a-paper-retracted-in-2022).
PS: I see that the [Stanford Encyclopedia of Philosophy](https://plato.stanford.edu/entries/induction-problem/#BayeSolu) has a page on this. I think it gives a good overview of the rigoristic camp, but it misses some of the points that a subjectivist would raise—e.g., that in cases with a wealth of data in favor of induction, like in the case of physical laws, the initial prior doesn't matter as long as it isn't "really crazy" (e.g., zero or extremely close to it), because one will reach pretty much the same conclusion of high certainty with any prior that isn't crazy.
[^1]: The reasonableness of requiring monotonicity might not be immediately apparent, so here is an explanation: The terms of that function are "plausibility of A" and "plausibility of B given that you know A is the case". By "monotonic", we mean that if we increase (resp. decrease) either of the inputs while keeping the other constant, or if we increase (resp. decrease) both at once, then the output—the plausibility of A and B—also has to increase (resp. decrease).
[^2]: We could also imagine living in a "maelstrom of chaos" universe in which past behavior is not predictive of future behavior. This could be the case at different levels of meta, i.e., it could also be difficult to predict whether future events are difficult to predict or understand. A real life example which bears some resemblance to this might be [Terry A. Davis](https://en.wikipedia.org/wiki/Terry_A._Davis), whose speech alternated between the utterings of a raving lunatic and gifted technical insights about programming.

View File

@ -0,0 +1,52 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBGMOGswBEAC1/EGI6ShrHYiJYlVwL99JviajFvXb1/rlLlbj1rIzQt9pEbyM
iTW8JBh3oTVbl6dXDweEWqI56myAc0OPl4hZZ9V7dm8C7lf6ul+8HW3vlTuD82pA
CBptGtaU0y8mgiATPD8evH1oWAIhTxS2GnhOAHl5QBjIr263/m/xjcNjO1qW/O7T
fhxFTCAkUnH/ZYtmVTVxKFshzwGHUX9gW0X2Yrt4wQ6bBOm3LsnbzRPuqMmiHR/Z
TYN0Y8JrTSWbAcoLojnnKj19IjwNEcM9hqeaN0MG0aNCUlADNVxEu72PyIdt4Ou8
fr707603DmTW62z18rsB7I0uR43rfoxd/MUp324/VSmS0R0G0TENf1mRtBmMhYn3
folj1taT0uXA+ybGN5KKlpt3yXMiKV4JukA/QwpxM/xpFAYiVc62VEYwFKNDBav+
G8O5x4nsenu5seD/j0iWmzqidISCJ+INwiAH5xvCZf6UpkvjnYgy3qj2cP3lns0c
Tnl6pIMlXvXK1TV2CGI3m8TDIdFm3dYGOHv6/UQ8C8x0qAUUdOcjPLuvd263VHzu
fUfn9JqIbHlaRv9Mk9bxoKM3SunitMDzOti40NzkvjQ9hmYwjbyZtrH/35Ryi++j
/vfde5Pr9uKhnhobMGiroS38d9WYqS1Qreh1bZtQzEGV3/4wfCxRdnyshwARAQAB
tDpOdcOxbyBTZW1wZXJlIChzb2NpYWwga2V5KSA8bnVuby5zZW1wZXJlbGhAcHJv
dG9ubWFpbC5jb20+iQJOBBMBCgA4FiEEzuFDyn7qGV+ATqFI/eT7w1dD2oQFAmMO
GswCGwMFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQ/eT7w1dD2oRQ9Q/+J0mn
/hwaQAsq98h/P9TyzIAnL4VZzTLuOBRqHCCSLz6LmwuN9vqDhtgA0rGaU2nyBZ2T
7RqBH8K3ZYzIWCR9dVngD9rqLtkGvbwSoBO5sptKsQOoZPgRq85UeQBWbLbk/7Vr
DoDpSGFJXWlXqeBx6yiPwFzuxd5P6zNHN3TALYEYI5ttDFcLi1wfdAeV453d/q2k
LPzvFSZw1BZO1gUjLYKEg8SvLnsCxTdSi1Z3MwPBqqOcH9ikOZfi0qaD6ZhruxET
q0QYT+VU/uyTYEOemWuTxc/khL1woHKEgSHBFHojKwzEs4gtkM6KTLg1KO8q+74T
Cd9t/PnDA1nvF59RqTJGBf6c74lyOiE2Lzp3hbkRyuVEMSmD3zFPGtvlArpWYdY8
xq7Hf5OIhn9fx7eQ7nPEjYuxBhL9LVricZkT3CBZopZn+vP+16nXLxGXfIW9nos2
1wuteBPa3RLn2XZfDapzMfZ0AwHt203Q16y34CYWeULnont/zkMXnLCKGzfSZSF+
lq+U0Al2E+EdMkoACU1sujF17YKpQs6DjmAGubuqCSRewonccgrXUj2+4FRTCzJC
rxI1JpOlmIbVVNDrbZTqOE6PmpSwnuDYqz/LYEBPGBnDSRC4yZ3+M2MXphSdBLFn
jEkzWa2tGNZZZN4CooUyhFIvBHRj2CO5O/69BM25Ag0EYw4azAEQAMF9++GpQgFC
vQaImsZ88y5SqMEn3Qb2sSeAzN8a10wHgz5LKKUB//eg4dHcvbZR0f2amiCSeMwE
XCPB8SGc1cmQ8GTK/Aw9ISZB1mTLntPOMduLP0QcX4r9vnUmbZ332ZBkoGUFOs1l
bmg9BxVYirvbzZ7KwHSXMsRLOpnYs6mwgjZtZfQhf+LA+40D0MALdgpLkIEKq765
+ZQm4dpjxHplsF7Ld8KhmxSs0eJ2IP/qC2QWeXGRVn+nHFKqm/difIMY8Uxyec1c
hwoMEXejF5mf5VA8ifCDKOjomdAvyQpJ9YpbCIRLzcrJrMK+bU1g19z9Msaq8mgY
PKPLaxrfWGIsHz16VASdE69HByldusFRXTxYRyODX0DdUWEM0mA3Gfu39Bjbbb1S
LxBrUjWxniOrR/dsLKxS++S1uiV8Gg3qZ8AWrJcEHfoWhtKnReEvjuOdyCtzhrWF
HAm48MlQzZ4Hxf0Tyoj2brgTXdF3njjr4x8J32egxhMh8nR43QYhA45UkqApEjYZ
WLWZKCAs5GEIJ5EIJM11Gw0CpCxabLTHLVxMSyHR4ZLemlzvaWoJYN5utRcGw+bJ
fNL0gPhEVPk788RD7OilCE7DR8AuxlnYQlL8GgPkvWX86YesyFnfq/FsbKEr+nAW
yVfgT0BFoB/0qcD9vw5HguI3dCjqfsbrABEBAAGJAjYEGAEKACAWIQTO4UPKfuoZ
X4BOoUj95PvDV0PahAUCYw4azAIbDAAKCRD95PvDV0PahJ1UD/9EbyXSNO42KiIV
6I6FWIvl7oeGyICop1Xnz14YUUTNKwLRdUwGEAMGYQeLSe2UIeaXKdlk1zHy5eQH
R9/cZxNkyAFqxN/zp0vwn4Riypk4vsTf2JohYyKLyBq1OX4k6r4iJsa+GqCMKbE0
ZDnSNAYR06mve4+8s0dgXRBwJTxZ41e7S8kIiZggKiOvwgTKB+2Ljp+3fVY0KjR4
VbmcEscP8rwJRX3+q9gVyTHMhPay1h6s9XisMQMV3600e7H1ct7mXA1SQ9eo/Esg
C1UaPPq/uQ67dlKABGlw+g3AsMmYf//bzn6TO2y/b2oXhi0yeVC20FHwVlid2N62
Z+0m6EHlan6lUMEEde9/Wh+fi6l5Ecn8wOVhG8IR91bsBYc9PCuRNWgK9BaLjZkN
7feTXlMkSeQaDCLZh5XrWrhbi/Z7lHC8pLiv6kgqYtDM6wkq0K73tfnczzivlim5
cNDsGv/mly1Cz4kfcGh2HlKh8C72p/qP+85U7YOd83R0okypVWVoeVbOfGyvkwHb
5cCX/oDFIUoGO2OW7yiK6NbaZuMcRxgPGsKr9rD6914CYuVSGETZ3Ef+jwiO1Tan
oBbY6OJUlhEUXDnmWRNdvB2Pj5NDRCCc34y/W/jNpMgTohqHsK7KroIu9PEZ6w+s
3uAqFODlDYNPXtA+CY1q/5q+KYeUNQ==
=s4xf
-----END PGP PUBLIC KEY BLOCK-----

View File

@ -1,3 +1,5 @@
_2022/08/30_: Per [this blogpost](https://forum.effectivealtruism.org/posts/tekdQKdfFe3YJTwML/end-to-end-encryption-for-ea#comments), I'm making my public key available [here](https://nunosempere.com/gossip/.nuno-sempere-public-key.txt). I can be contacted at nuno dot sempere dot lh at protonmail dot com.
_2022/08/14_: Updated [blog rss](https://nunosempere.com/blog/index.rss) to use the correct date. Thanks, kind stranger who left a tip on my [Admonymous](https://www.admonymous.co/loki). _2022/08/14_: Updated [blog rss](https://nunosempere.com/blog/index.rss) to use the correct date. Thanks, kind stranger who left a tip on my [Admonymous](https://www.admonymous.co/loki).
_2022/07/05_: I am offering [cancellation insurance](https://nunosempere.com/blog/2022/07/04/cancellation-insurance/) and offering [to bet on the success or failure of projects](https://nunosempere.com/blog/2022/07/05/i-will-bet-on-your-success-or-failure/). _2022/07/05_: I am offering [cancellation insurance](https://nunosempere.com/blog/2022/07/04/cancellation-insurance/) and offering [to bet on the success or failure of projects](https://nunosempere.com/blog/2022/07/05/i-will-bet-on-your-success-or-failure/).

1
misc/test/test-git Submodule

@ -0,0 +1 @@
Subproject commit 4f5fa42a8214057289b30ff92ef5fe082700d59e

2
misc/test/test-git.ash Normal file
View File

@ -0,0 +1,2 @@
cd /home/www/werc/werc-1.5.0/sites/nunosempere.com/misc/test/test-git
git pull github master

Binary file not shown.

View File

@ -188,10 +188,25 @@ https://nunosempere.com/blog/2022/07/23/
https://nunosempere.com/blog/2022/07/23/thoughts-on-turing-julia/ https://nunosempere.com/blog/2022/07/23/thoughts-on-turing-julia/
https://nunosempere.com/blog/2022/07/27/ https://nunosempere.com/blog/2022/07/27/
https://nunosempere.com/blog/2022/07/27/how-much-to-run-to-lose-20-kilograms/ https://nunosempere.com/blog/2022/07/27/how-much-to-run-to-lose-20-kilograms/
https://nunosempere.com/blog/2022/08/
https://nunosempere.com/blog/2022/08/04/
https://nunosempere.com/blog/2022/08/04/usd1-000-squiggle-experimentation-challenge/
https://nunosempere.com/blog/2022/08/08/
https://nunosempere.com/blog/2022/08/08/forecasting-newsletter-july-2022/
https://nunosempere.com/blog/2022/08/10/
https://nunosempere.com/blog/2022/08/10/evolutionary-anchor/
https://nunosempere.com/blog/2022/08/18/
https://nunosempere.com/blog/2022/08/18/what-do-americans-mean-by-cutlery/
https://nunosempere.com/blog/2022/08/20/
https://nunosempere.com/blog/2022/08/20/fermi-introduction/
https://nunosempere.com/blog/2022/08/31/
https://nunosempere.com/blog/2022/08/31/on-cox-s-theorem-and-probabilistic-induction/
https://nunosempere.com/forecasting/ https://nunosempere.com/forecasting/
https://nunosempere.com/gossip/ https://nunosempere.com/gossip/
https://nunosempere.com/misc/ https://nunosempere.com/misc/
https://nunosempere.com/misc/poetry/ https://nunosempere.com/misc/poetry/
https://nunosempere.com/misc/test/ https://nunosempere.com/misc/test/
https://nunosempere.com/misc/test/test-git/
https://nunosempere.com/misc/test/test-git/README
https://nunosempere.com/research/ https://nunosempere.com/research/
https://nunosempere.com/software/ https://nunosempere.com/software/