feat: add other posts

blog/2022/09/15/use-distributions-to-more-parsimoniously-estimate-impact/index.md

@@ -0,0 +1,102 @@
+Use distributions to more parsimoniously estimate impact
+========================================================
+
+## Summary
+
+By incorporating uncertainty into its estimates, GiveWell would produce better estimates. This is best done by working with distributions, as opposed to point estimates. For example, "$294 per doubling of consumption" is a point estimate[^1], but the following is a distribution:
+
+![](https://i.imgur.com/jEP0tE9.png)
+
+This post is an entry to GiveWell's [Change Our Mind Contest](https://www.givewell.org/research/change-our-mind-contest).
+
+## Index
+
+- Summary
+- Prior work
+- Advantages
+  - Bottom line may change
+  - Value of information calculations become easier
+  - it becomes possible to (attempt to adjust) for the optimizers' curse
+  - It is the parsimonious way to do this, and thus I expect a better treatment of uncertainty to have further accuracy gains
+- How to do it
+  - Which language to use
+  - To do it in-house or to delegate it
+  - To do it all at once or bit by bit
+- Conclusion
+
+
+## Prior work
+
+People have been pointing out this problem for a while:
+
+- [Uncertainty and sensitivity analyses of GiveWell's cost-effectiveness analyses](https://forum.effectivealtruism.org/posts/gMxTEMvh8RttX9Nt4/uncertainty-and-sensitivity-analyses-of-givewell-s-cost)
+- [Quantifying Uncertainty in GiveWell's GiveDirectly Cost-Effectiveness Analysis](https://forum.effectivealtruism.org/posts/4Qdjkf8PatGBsBExK/adding-quantified-uncertainty-to-givewell-s-cost)
+- [Adding Quantified Uncertainty to GiveWell's Cost Effectiveness Analysis of the Against Malaria Foundation](https://forum.effectivealtruism.org/posts/4Qdjkf8PatGBsBExK/adding-quantified-uncertainty-to-givewell-s-cost)
+- [A critical review of GiveWell's 2022 cost-effectiveness model](https://forum.effectivealtruism.org/posts/6dtwkwBrHBGtc3xes/a-critical-review-of-givewell-s-2022-cost-effectiveness)
+
+This is because adding distributional estimates is relatively low-hanging fruit that GiveWell may have been procrastinating on.
+
+## Advantages
+
+### Bottom line may change
+
+The key reason to use distributional estimates is that the bottom line may change.
+
+One key example I keep pointing to is the paper [Dissolving the Fermi Paradox](https://arxiv.org/abs/1806.02404). They provide a clear example where using point estimates leads to a very different outcome than using distributions. When using point estimates, one arrives at the [Fermi paradox](https://en.wikipedia.org/wiki/Fermi_paradox), whereas when using distributions one arrives at the reasonable conclusion that we have a high chance of being alone in the galaxy.
+
+Similarly, GiveWell's estimates may be different if they use distributional estimates, particularly for distributions for which there is high uncertainty. This might be the case for individual interventions such as deworming.
+
+### Value of information calculations become easier
+
+With distributions, value of information calculations become more tractable. In particular, the probability that a given intervention will beat GiveWell's top charities upon further investigation becomes a calculable amount. This amount would depend on the current distribution of impact of the new opportunity, the expected narrowing with more investigation, the distribution of impact of current GW top charities, and the expected yearly amount of funding.
+
+This point might might also affect meta-global-health and development: better treatment of uncertainty might suggest that incubators, meta-charities, etc., such as [IPA](https://www.poverty-action.org/) or [Charity Entrepreneurship](https://www.charityentrepreneurship.com/) are more or less neglected than currently considered. It would also ground the need for something like a [GiveWell incubation grant](https://www.givewell.org/research/incubation-grants) in terms of a new charity having a high enough chance of beating GiveWell charities, which could be calculated from its current expected impact distribution.
+
+### It becomes possible to (attempt to adjust) for the optimizers' curse
+
+The [Optimizers' curse](https://christiansmith.cc/2019/04/03/the-optimizers-curse-wrong-way-reductions/) seems like the biggest theoretical objection to GiveWell's current recommendations. It points out that opportunities estimated to be the best will be selected for having high impact, but also for having a high error rate in the estimation process. It's possible that this may have been the case for deworming.
+
+The Optimizers' curse is not unfixable, and in particular there is a [Bayesian solution](https://www.lesswrong.com/posts/5gQLrJr2yhPzMCcni/the-optimizer-s-curse-and-how-to-beat-it). But for this solution to work, one has to construct a prior[^2], and the estimates have to be in distributional form.
+
+Note that GiveWell may have previously been bit by the optimizers' curse: in [their answer to HLI on deworming, they write](https://forum.effectivealtruism.org/posts/MKiqGvijAXfcBHCYJ/deworming-and-decay-replicating-givewell-s-cost?commentId=Qt26uR9ZT6ru8xDqi#comments), they write:
+
+> Once we do this work, our best guess is that we will reduce our estimate of the cost-effectiveness of deworming by 10%-30%. Had we made this change in 2019 when KLPS-4 was released, we would have recommended $2-$8m less in grants to deworming (out of $55m total) since 2019.
+
+Correcting for this by explicitly accounting for the optimizers' curse may have made this error less severe—though GiveWell probably did adjust, if perhaps on a more ad-hoc way. See some discussion of a related point [here](https://forum.effectivealtruism.org/posts/MKiqGvijAXfcBHCYJ/deworming-and-decay-replicating-givewell-s-cost?commentId=mTWzsjgDhGCyAu24z).
+
+### It is the parsimonious way to do this, and thus I expect a better treatment of uncertainty to have further accuracy gains
+
+Ultimately, the "true shape" of impact estimates—and of our knowledge about the world more generally—seems like it is uncertain, probabilistic, and distributional. As such, I expect there would be additional advantages besides the ones I've pointed above, like:
+
+- More easily identifying the parts of a calculation which are more crucial to its result
+- Allowing donors to make decisions which better fit their own risk profiles
+- Making estimates, such as the ones for AMF which depend on the age of the beneficiary, way less clunky[^3]
+- Comparisons against other cause areas (e.g., animal welfare) may become a bit more accurate
+
+## How to do it
+
+### Which tools to use
+
+Because of GiveWell's model of doing its analysis and then publishing it, I think that mostly any programming language with support for manipulating probabilities (R, Python, Turing.jl, Stata, etc.) would be ok. In particular, the key factor seems like ease of use for current GiveWell staff-members.
+
+I'd put in a good word for [Squiggle](https://www.squiggle-language.com/), the language which my org is developing. It may be particularly suitable for making calculators or tools to allow users to later tweak estimates in the browser.
+
+But I think that any choice of tool that supports distribution would do a-ok.
+
+### To do it in-house or to delegate it
+
+I think that doing this kind of thing well would require having plenty of access to GiveWell's internal knowledge and access to GiveWell staff's intuitions. And an outside effort would have to be so tightly integrated with the GiveWell team that you might as well do it in-house.
+
+### To do it all at once or bit by bit
+
+It is possible that translating all models to distributions might be too much work for one year—I can't say since I'm not familiar with GiveWell's internal procedures. Because of the uncertainty in how long this would take, it seems to me that creating a distributional estimate alongside the traditional pointwise estimate for one top charity next year seems like a prudent way to start out.
+
+## In conclusion
+
+In conclusion, I suggest that GiveWell should be creating distributional rather than pointwise estimates, because this would lead to more accurate results and a better treatment of uncertainty.
+
+[^1]: And in fact, I think this corresponds to GiveWell's current estimate, obtained by dividing over [this cell](https://docs.google.com/spreadsheets/d/1Kq6iHSQFr3eRz1p9KclHuJTQiaJYkpViOyneSd9KCJc/edit#gid=1680005064), because one "unit of value" is one doubling of consumption for one year.
+
+[^2]: One shortcut when considering a _new_ charity might be to use a guess as to the distribution of cost-effectiveness for all past charities GiveWell has considered in the past.
+
+[^3]: Right now, AMF assigns a homogeneous value to lives within a bucket of ~5 years, and then estimates what proportion of recipients belong to each bucket. This is complicated and clunky; one could instead do something like what's hinted [here](https://forum.effectivealtruism.org/posts/4Qdjkf8PatGBsBExK/adding-quantified-uncertainty-to-givewell-s-cost?commentId=q6EQPSmMCg8cTiEDN), i.e., define continuous functions for each of the components of the estimation, and then combine them. In particular, the value of a life of age _t_ can be estimated as the life expectancy for someone that age, without the need for bucketing.

blog/2022/09/19/utilitarianism-an-incomplete-approach/index.md

@@ -0,0 +1,75 @@
+Utilitarianism: An Incomplete Approach
+======================================
+
+This blog post gives the sketch of a book, or maybe a long article, that's been on my mind for a while. I wrote it last week, over the course of an hour an a half, with _The Incredibles_ blasting on the background and with me feeling intellectually alive.
+
+### Chapter 1. Utilitarism: The building blocks
+
+This chapter would define utilitarianism, and go over the building blocks of expected utility maximization, like I did [in this post](https://forum.effectivealtruism.org/s/XbCaYR3QfDaeuJ4By/p/8XWi8FBkCuKfgPLMZ) but without boring the reader to death. The building blocks are:
+
+<img src="./.images/argmax.png" class="img-medium-center">
+
+1. The utility function
+2. The set of actions to choose from
+3. The expected value function
+4. The knowledge of the world we condition over
+5. The method we use to choose between the sets of actions
+
+We can't always talk about these independently, and there are observations that apply to these parts in combination. From where I'm standing, I think I have to say most things to say about points 1., 3., 4., and maybe 5.
+
+### Chapter 2: The utility function
+
+Go over the basis of why the shape of some of your goals and desires might be utilitarian, or at least consequentialist.
+
+Start with the [von Neumann-Morgenstern utility theorem](https://en.wikipedia.org/wiki/Von_Neumann%E2%80%93Morgenstern_utility_theorem), and go over various ways to stare at expected utility maximization in the face. Go over Harsany's proof of utilitarianism. Go over [what goes wrong if you try to discount small probabilities](https://petrakosonen.files.wordpress.com/2022/07/chapter-4-how-to-discount-small-probabilities.pdf).
+
+There are two threads of arguments here. One thread is the vast literature on this topic, which to my uninitiated eyes seems like it goes around in circles a bit. The other thread are the intuitive reasons that utilitarianism has going for it. Ultimately, we can point to reasons for having utilitarian intuitions. Then for the rest of the text we can assume utilitarianim in theory, and ask about how to implement it in practice. And we can circle back to philosophical critiques at the theoretical level when considering limitations in the last chapter.
+
+### Chapter 3: The set of actions
+
+Point 2., the set of actions to choose from, seems more dependent on the time and place. For example, the 80,000 hours recommendations have changed in the last half a decade.
+
+Still, maybe we can talk about robust ways of improving the set of actions one has access to, in order to do the most good. Or about [heuristics](https://forum.effectivealtruism.org/posts/EP6X362Q3ziibA99e/show-a-framework-for-shaping-your-talent-for-direct-work) and instrumental goals, and see what other long-lived organizations have done to attain success. Honestly not sure what to say here, I haven't been that much of a man of action.
+
+### Chapter 4: Expected value and the knowledge behind it
+
+Expected value can be divided into:
+
+1. estimations of value of different states, and
+2. estimation of probabilities.
+
+For probabilities, we can bring in Bayesian probability theory on the theory side, and forecasting on the practical side. Discuss [Cox's theorem](https://en.wikipedia.org/wiki/Cox's_theorem), because I want to and because I think it's really elegant. Then discuss methods and trends in forecasting. Maybe add a bit of color by going through some nice predictions I've done with Samotsvety.
+
+For values, we can go over how QALYs are created. And then we can try to generalize this. One could do this with relative values, but I think I'm more optimistic about writting down the pathway to impact directly and trying to estimate impact in general units, like QALYs or "basis points of the future"
+
+### Chapter 5: The method of choice
+
+Normally the method of choice would be to argmax, i.e., to choose the best option. You could also do quantilization (choose something amonst the top p% (e.g., top 5%) of options at randomly. Discuss Goodhart's law and the perils of maximizing on observables. Maybe discuss Pascal's mugging.
+
+### Chapter 6: The limitations
+
+Above, note that I didn't argue that one should be purely utilitarian, but rather that the altruistic parts of yourself should take that shape.
+
+Then discuss:
+
+- what can go wrong if you try to fit your preferences to a utility function.
+- what can go wrong if you use expected values
+- what can go wrong if you maximize too hard.
+- classical objections to utilitarianism (repugnant conclusion &c).
+- etc.
+
+The limitations of my own perspective are that I'm kind of a jack of all trades, idiosyncratically deeply familiar with some things, but not with others.
+
+### Chapter 7: Answers to limitations
+
+1. The cop out answer is to be rule utilitarian
+2. One could also be an "incrementalist": Argue for pushing the envelope. In combination with other perspectives, utilitarianism can be incredibly powerful, and it's a perspective worth listening to.
+3. There is also bullet-biting utilitarianism maximalism.
+
+### Chapter 8: These ideas in practice
+
+Discuss how I've applied these ideas, at the Quantified Uncertainty Research Institute & in other places. Maybe discuss these ideas in relation to Effective Altruism as practiced today.
+
+### Conclusion
+
+In conclusion, people IMHO are quick to see the flaws and slow to see the appeal of a more hardcore type of utilitarianism. And the text of whcih the above is a very rough sketch could fill that gap.