+ ",,"★★★☆☆"
+"Results for Development — Childhood Pneumonia Treatment Program (2019)","https://www.givewell.org/research/incubation-grants/results-for-development/january-2019-grant#Internal_forecasts","GiveWell",false,"none","
Internal forecasts
For this grant, we are recording the following forecasts:
| Confidence | Prediction | By time |
|---|
| 40% | R4D or an R4D program is a top charity | December 2023 |
| 35% | R4D or an R4D program is a top charity and we estimate that donations to that program are at least half as cost-effective as the most cost-effective unfunded giving opportunity among top charities (i.e. where we recommend donors give on the margin) | December 2023 |
| 5% | R4D or an R4D program is a top charity and we estimate that donations to that program are at least twice as cost-effective as the most cost-effective unfunded giving opportunity among top charities (i.e. where we recommend donors give on the margin) | December 2023 |
",,"★★★☆☆"
+"Charity Science Health — Exit Grant","https://www.givewell.org/research/incubation-grants/charity-science-exit-grant-july-2019#Internal_forecasts","GiveWell",false,"none","
Internal forecasts
For this grant, we are recording the following forecast:
| Confidence | Prediction | By time |
|---|
| 65% | Charity Science Health receives enough funding from other donors to continue its operations through the end of 2020. | End of 2020 |
",,"★★★☆☆"
+"Innovations for Poverty Action — Randomized Controlled Trial on the","https://www.givewell.org/research/incubation-grants/innovations-for-poverty-action-masks-rct-july-2020#Internal_forecasts","GiveWell",false,"none","
Internal forecasts
We made a number of forecasts related to this grant. These forecasts were made by the lead grant investigator and were informed by the material contained in this grant page, as well as intuitive reasoning. Because of the uncertainty around how this grant will resolve, we felt that making explicit forecasts would help us to determine the extent to which we exercised good judgment in making this grant after the fact. The resolution dates listed below are the dates by which we expect to evaluate the predictions associated with them.
| Prediction | Credence | Resolution date |
|---|
| The RCT will NOT find a statistically significant32 result on self-reported respiratory symptoms, AND it will not find a statistically significant result on COVID-19.33 | 15% | June 1, 2021 |
| The RCT will find a statistically significant result on self-reported respiratory symptoms, but NOT COVID-19. | 20% | June 1, 2021 |
| The RCT will NOT generate a statistically significant result on AT LEAST ONE of the outcomes, and the authors will say in their first preprint that an important reason was lower than expected incidence of COVID-19 or self-reported respiratory symptoms. | 25% | June 1, 2021 |
| The RCT will NOT generate a statistically significant result on AT LEAST ONE of the outcomes, and the authors will say in their first preprint that an important reason was a failure to increase mask-wearing. | 20% | June 1, 2021 |
| Results on respiratory symptoms (Conditional on a paper being published in a top 200 academic journal) |
|---|
| The RCT will NOT find a statistically significant (p>0.05) impact on self-reported respiratory symptoms. | 13% | June 1, 2021 |
| Statistically significant result and reduction in respiratory symptoms of 0-10%. | 5% | June 1, 2021 |
| Statistically significant result and reduction in respiratory symptoms of 10-20%. | 20% | June 1, 2021 |
| Statistically significant result and reduction in respiratory symptoms of 20-30%. | 30% | June 1, 2021 |
| Statistically significant result and reduction in respiratory symptoms of 30-40%. | 22% | June 1, 2021 |
| Statistically significant result and reduction in respiratory symptoms of >40%. | 10% | June 1, 2021 |
| Results on COVID-19 outcome (Conditional on a paper being published in a top 200 academic journal) |
|---|
| The paper will not report an objectively measured disease outcome confirmed with diagnostic tests. | 10% | June 1, 2021 |
| Conditional on a COVID-19 outcome being reported in the published paper |
| The RCT will NOT find a statistically significant (p>0.05) impact on preferred measure of COVID-19. | 25% | June 1, 2021 |
| Statistically significant result and reduction in COVID-19 of 0-10%. | 5% | June 1, 2021 |
| Statistically significant result and reduction in COVID-19 of 10-20%. | 14% | June 1, 2021 |
| Statistically significant result and reduction in COVID-19 of 20-30%. | 28% | June 1, 2021 |
| Statistically significant result and reduction in COVID-19 of 30-40%. | 20% | June 1, 2021 |
| Statistically significant result and reduction in COVID-19 of >40%. | 8% | June 1, 2021 |
| Attention as a path to impact |
|---|
| At least one academic paper will be published in a top 200 academic journal on the basis of this RCT. | 85% | June 1, 2021 |
| Conditional on the study getting a statistically significant result on EITHER respiratory symptoms OR COVID-19, it will be mentioned in over 50,000 tweets according to altmetrics. | 80% | June 1, 2021 |
| Conditional on a preprint being published (i.e., a draft paper is published on the internet), the study will receive justified criticism (as bad as this; determined by a panel of three GiveWell researchers if it's not obvious) questioning its conclusions. | 1% | June 1, 2021 |
| Impact |
|---|
| Conditional on the study getting a statistically significant result on EITHER respiratory symptoms OR COVID-19, the WHO will publish an update to its guidance on masks mentioning this study. | 65% | June 1, 2021 |
| Conditional on the study getting a statistically significant result on EITHER respiratory symptoms OR COVID-19, we will see at least one country with a population >20 million update its guidance on mask-wearing, explicitly referring to EITHER this evidence OR updated WHO guidance that refers to this evidence. | 60% | June 1, 2021 |
| Conditional on the study getting a statistically significant result on EITHER respiratory symptoms OR COVID-19, it will be mentioned in the minutes of the UK's SAGE discussions. | 40% | June 1, 2021 |
| Timing | |
|---|
| A preprint is published (i.e., a draft paper is published on the internet) on the basis of this RCT by October 1, 2020. | 20% | October 1, 2020 |
| A preprint is published (i.e., a draft paper is published on the internet) on the basis of this RCT by December 1, 2020. | 55% | December 1, 2020 |
| A preprint is published (i.e., a draft paper is published on the internet) on the basis of this RCT by February 1, 2021. | 65% | February 1, 2021 |
| A preprint is published (i.e., a draft paper is published on the internet) on the basis of this RCT by April 1, 2021. | 75% | April 1, 2021 |
",,"★★★☆☆"
+"Center for Effective Global Action at UC Berkeley — Scoping RCTs for","https://www.givewell.org/research/incubation-grants/cega-uc-berkeley/july-2017-grant#Internal_forecasts","GiveWell",false,"none","
Internal forecasts
For this project, we are recording the following forecasts. Note that these forecasts assume two postdoctoral researchers assigned to the project by CEGA.
- 40% chance that by the end of 2019 we will fund at least one long-term follow-up study because of this project.
- 20% chance that by the end of 2019 we will fund at least two long-term follow-up studies because of this project.
- 35% chance that by the end of 2019 we renew the grant to fund further research into intervention areas where we could potentially fund long-term follow-up studies.
- 5% chance that by the end of 2020 we will fund at least five long-term follow-up studies because of this project.
- 40% chance that by the end of 2020 non-GiveWell funder(s) will fund at least three long-term follow-up studies because of this project.
- 5% chance that by the end of 2020 a long-term follow-up study funded because of this project (by us or another funder) will affect how we allocated more than $10 million in funding from 2018-2020.
- 25% chance that by the end of 2025 a long-term follow-up study funded because of this project (by us or another funder) will affect how we allocate more than $10 million in funding from 2018-2025.
- 10% chance that by the end of 2025 a long-term follow-up study funded because of this project (by us or another funder) will affect how we allocate more than $40 million in funding from 2018-2025.
",,"★★★☆☆"
+"Evidence Action Beta — Incubator Program","https://www.givewell.org/research/incubation-grants/july-2018-evidence-action-beta-incubator#Internal_forecasts","GiveWell",false,"none","
Internal forecasts
For this grant, we are recording the following forecasts:
| Confidence | Prediction | By Time |
|---|
| 30% | This grant does not lead to any new top charities. | December 2023 |
| 55% | The Beta incubator leads to a new top charity that is 1-2x the cost-effectiveness of our marginal spending on current top charities. | December 2023 |
| 10% | The Beta incubator leads to a new top charity that's >2x as cost-effective as our marginal spending on current top charities | December 2023 | |
| 5% | The Beta incubator program has impacts that lead us to make a public case that it was extremely cost-effective overall (i.e., it resulted in at least $10 million in spending at 15x the cost-effectiveness of cash transfers or more). | December 2023 |
| 15% | Our marginal spending on top charities will be 2.5x as cost-effective as cash or less (using our current cost-effectiveness estimate for cash) | December 2023 |
",,"★★★☆☆"
+"Centre for Pesticide Suicide Prevention — General Support","https://www.givewell.org/research/incubation-grants/centre-pesticide-suicide-prevention/august-2017-grant#Internal_forecasts","GiveWell",false,"none","
Internal forecasts
For this grant, we are recording the following forecasts:
| Confidence | Prediction | By Time |
|---|
| Elie: 67%; James: 65% | We recommend a second grant to CPSP | July 1, 2019 |
| Elie: 33%; James: 55% | Conditional on CPSP entering Nepal, Nepal will pass legislation to ban at least one of the three pesticides most commonly used in suicide. | July 1, 2020 |
| Elie: 5%; James: 10% | Conditional on CPSP entering India, India will pass legislation to ban at least one of the three pesticides most commonly used in suicide. | July 1, 2020 |
| Elie: 15%; James: 35% | Conditional on CPSP entering India, a state in India will pass legislation to ban at least one of the three pesticides most commonly used in suicide. | July 1, 2020 |
| James: 85% for each | Conditional on legislation to ban pesticides used in suicide being passed in Nepal / India, the pesticide suicide rate is lower in the combined three years after the ban than it is in the three years before the ban (as determined by national level data or representative survey data) | 3 years after legislation passed |
| James: 40% for each | Conditional on legislation to ban pesticides used in suicide being passed in Nepal / India, the pesticide suicide rate is >10% lower in the combined three years after the ban than it is in the three years before the ban (as determined by national level data or representative survey data) | 3 years after legislation passed |
| James: 20% for each | Conditional on legislation to ban pesticides used in suicide being passed in Nepal / India, the pesticide suicide rate is >15% lower in the combined three years after the ban than it is in the three years before the ban (as determined by national level data or representative survey data) | 3 years after legislation passed |
",,"★★★☆☆"
+"Innovation in Government Initiative — General Support","https://www.givewell.org/research/incubation-grants/innovation-in-government-initiative/december-2018-grant#Internal_forecasts","GiveWell",false,"none","
Internal forecasts
For this grant, we are recording the following forecasts:
| Confidence | Prediction | By Time |
|---|
| 35% | GiveWell makes another grant to IGI | January 1, 2021 |
| 60% | IGI is able to raise more than $1 million in funding from other sources | January 1, 2021 |
",,"★★★☆☆"
+"IDinsight — Beneficiary Preferences Survey (2019)","https://www.givewell.org/research/incubation-grants/IDinsight-beneficiary-preferences-march-2019#Internal_forecasts","GiveWell",false,"none","
Internal forecasts
For this grant, we are recording the following forecast:
| Confidence | Prediction | By time |
|---|
| 35% | At least four GiveWell staff members with inputs in our cost-effectiveness model change their moral weights for either valuing health vs. income or age-weighting by at least 25%, and they attribute that change to this research | End of 2020 |
",,"★★★☆☆"
+"Fortify Health — General Support (2019)","https://www.givewell.org/research/incubation-grants/fortify-health/august-2019-grant#Internal_forecasts","GiveWell",false,"none","
Internal forecasts
For this grant, we are recording the following forecasts:
| Confidence | Prediction | By time |
|---|
| 25% | Fortify Health becomes a top charity. | November 2022 |
| 75% | Fortify Health has successfully signed agreements to install equipment with at least an additional 8 mills. | July 2020 |
| 25% | Fortify Health has successfully signed agreements to install equipment with at least an additional 16 mills. | July 2020 |
| 33% | Fortify Health has successfully installed equipment in at least 8 additional partner mills. | July 2020 |
| 10% | Fortify Health has successfully installed equipment in at least 16 additional partner mills. | July 2020 |
| 90% | Fortify Health successfully maintains its four existing miller partnerships. | July 2020 |
| 60% | Laboratory tests from random samples of atta produced by Fortify Health's partner mills do not fall more than 1mg below the target (21.25 mg of iron per kilogram of wheat flour) in more than 25% of cases. | July 2020 |
",,"★★★☆☆"
+"Fortify Health — General Support","https://www.givewell.org/research/incubation-grants/fortify-health/june-2018-grant#Internal_forecasts","GiveWell",false,"none","
Internal forecasts
For this grant, we are recording the following forecasts:
| Confidence | Prediction | By Time |
|---|
| 35% | GiveWell makes another grant to Fortify Health to work on iron fortification in India | May 1, 2019 |
| 50% | Fortify Health completes installation of fortification equipment in at least one mill | May 1, 2019 |
| 20% | Fortify Health completes installation of fortification equipment in at least five mills | May 1, 2019 |
| 10% | Fortify Health becomes a GiveWell top charity | May 1, 2022 |
| 60% | GiveWell models Fortify Health as more than 10x as cost-effective8 as cash after updating our CEA based on the Cochrane review of iron fortification that is scheduled to be released in 2018 | May 1, 2019 |
",,"★★★☆☆"
+"New Incentives — General Support (April 2020)","https://www.givewell.org/research/incubation-grants/new-incentives/april-2020-grant#Internal_forecasts","GiveWell",false,"none","
Internal forecasts
For this grant, we are recording the following forecast:
75% chance that GiveWell will make a decision by September 30, 2020, about whether to recommend that Open Philanthropy and other donors continue to fund New Incentives.
",,"★★★☆☆"
+"Harvard University — Solar Geoengineering Research Program","https://www.openphilanthropy.org/focus/global-catastrophic-risks/miscellaneous/harvard-university-solar-geoengineering-research-program","GiveWell",false,"none","
Internal forecasts
+
We are experimenting with recording explicit numerical forecasts of the probability of events related to our decision-making (especially grant-making). The idea behind this is to pull out the implicit predictions that are playing a role in our decisions, and to make it possible for us to look back on how well-calibrated and accurate those predictions were. For this grant, we are recording the following public forecast:
+
+- 80% chance that we will consider this grant a success in 10 years.
+
+
",,"★★★☆☆"
+"UC Berkeley — Center for Human-Compatible AI (2016)","https://www.openphilanthropy.org/focus/global-catastrophic-risks/potential-risks-advanced-artificial-intelligence/uc-berkeley-center-human-compatible-ai","GiveWell",false,"none","
About the grant
+
This grant will support the establishment of the Center for Human-Compatible AI at UC Berkeley, led by Professor Russell with the following co-Principal Investigators and collaborators:
+
+- Pieter Abbeel, Associate Professor of Computer Science, UC Berkeley
+
- Anca Dragan, Assistant Professor of Computer Science, UC Berkeley
+
- Tom Griffiths, Professor of Psychology and Cognitive Science, UC Berkeley
+
- Bart Selman, Professor of Computer Science, Cornell University
+
- Joseph Halpern, Professor of Computer Science, Cornell University
+
- Michael Wellman, Professor of Computer Science, University of Michigan
+
- Satinder Singh Baveja, Professor of Computer Science, University of Michigan
+
Research topics that the Center may focus on include:
+
+- Value alignment through, e.g., inverse reinforcement learning from multiple sources (such as text and video).
+
- Value functions defined by partially observable and partially defined terms (e.g. “health,” “death”).
+
- The structure of human value systems, and the implications of computational limitations and human inconsistency.
+
- Conceptual questions including the properties of ideal value systems, tradeoffs among humans and long-term stability of values.
+
We see the creation of this center as an important component of our efforts to help build the field of AI safety for several reasons:
+
+- We expect the existence of the Center to make it much easier for researchers interested in exploring AI safety to discuss and learn about the topic, and potentially consider focusing their careers on it. Ideally this will result in a larger number of researchers ending up working on topics related to AI safety than otherwise would have.
+
- The Center may allow researchers already focused on AI safety to dedicate more of their time to the topic and produce higher-quality research.
+
- We hope that the existence of a well-funded academic center at a major university will solidify the place of this work as part of the larger fields of machine learning and artificial intelligence.
+
Based on our in-progress investigation of field-building, our impression is that funding the creation of new academic centers is a very common part of successful philanthropic efforts to build new fields.
+
We also believe that supporting Professor Russell’s work in general is likely to be beneficial. He appears to us to be more focused on reducing potential risks of advanced artificial intelligence (particularly the specific risks we are most focused on) than any comparably senior, mainstream academic of whom we are aware. We also see him as an effective communicator with a good reputation throughout the field.
+
Budget and room for more funding
+
Professor Russell estimates that the Center could, if funded fully, spend between $1.5 million and $2 million in its first year and later increase its budget to roughly $7 million per year.
+
Professor Russell currently has a few other sources of funding to support his own research and that of his students (all amounts are approximate):
+
+- $340,000 from the Future of Life Institute
+
- $280,000 from the Defense Advanced Research Projects Agency
+
- $1,500,000 from the Leverhulme Trust (spread over 10 years)
+
Our understanding is that most of this funding is already or will soon be accounted for, and that Professor Russell would not plan to announce a new Center of this kind without substantial additional funding. Professor Russell has also applied for a National Science Foundation Expedition grant, which would be roughly $7 million to $10 million over ten years. However, because we do not expect that decision to be made until at least a few months after the final deadline for proposals in January 2017, and because we understand those grants to be very competitive, we decided to set our level of funding without waiting for that announcement.
+
We are not aware of other potential funders who would consider providing substantial funding to the Center in the near future, and we believe that having long-term support in place is likely to make it easier for Professor Russell to recruit for the Center.
+
Internal forecasts
+
We’re experimenting with recording explicit numerical forecasts of events related to our decisionmaking (especially grantmaking). The idea behind this is to pull out the implicit predictions that are playing a role in our decisions, and make it possible for us to look back on how well-calibrated and accurate those are. For this grant, we are recording the following forecast:
+
+- 50% chance that, two years from now, the Center will be spending at least $2 million a year, and will be considered by one or more of our relevant technical advisors to have a reasonably good reputation in the field.
+
+
",,"★★★☆☆"
+"Compassion in World Farming USA — General Support (2016)","https://www.openphilanthropy.org/focus/us-policy/farm-animal-welfare/CIWF-USA-general-support","GiveWell",false,"none","
Plans for learning and follow-up
+
Goals and expectations for this grant
+
We hope this grant will enable CIWF USA to:
+
+- Grow into a role similar to that which we see organizations like the Humane Society of the United States (HSUS) filling, working cooperatively with U.S. corporations to achieve farm animal welfare reforms.
+
- Persuade at least one major poultry producer (e.g. Perdue Foods, Tyson Foods, Pilgrim’s Pride Corporation) to adopt a meaningful broiler welfare policy, thereby setting a precedent in the industry.
+
- Continue the existing trend toward cage-free eggs and begin securing broiler chicken welfare policies at major companies.
+
+
Internal forecasts
+
We’re experimenting with recording explicit numerical forecasts of events related to our decisionmaking (especially grantmaking). The idea behind this is to pull out the implicit predictions that are playing a role in our decisions, and make it possible for us to look back on how well-calibrated and accurate those are. For this grant, we are recording the following forecasts:
+
+- 50% chance that CIWF USA will persuade at least one major poultry company to adopt a meaningful broiler chicken welfare policy.
+
- 75% chance that CIWF USA will play a major role in securing five or more new corporate cage-free pledges.
+
",,"★★★☆☆"
+"Impact Justice — Restorative Justice Project","https://www.openphilanthropy.org/focus/us-policy/criminal-justice-reform/impact-justice-restorative-justice-project","GiveWell",false,"none","
Plans for follow-up
+
After the first year of the grant period, we plan to begin evaluating the progress of the grant by looking for the following markers of success:
+
+- Attempts by criminal justice officials to bring in practitioners from other places to work with them on further developing the concept of restorative justice.
+
- Public statements of support for restorative justice.
+
- Increasing use of language related to restorative justice.
+
- Political partners on restorative justice work stepping forward as champions or allies on other types of local criminal justice reform.
+
- Public recognition by federal-level system leaders that restorative justice is a viable and desirable alternative to incarceration.
+
+
Goals for the grant
+
We have two main goals for this grant:
+
+- To advance restorative justice as a potential alternative to incarceration in the long term. In particular, we believe that restorative justice could be a useful alternative in the types of cases that are unlikely to benefit from the current political consensus on the need for drug law and mental health reform. Our understanding is that restorative justice has been shown to work best in the serious and violent cases for which most other criminal justice reform efforts have failed.
+
- To change the political culture in the targeted jurisdictions to be more open to criminal justice reform in general.
+
+
Key questions for follow-up
+
+- How many key stakeholders is Impact Justice engaging in each location? How far along are they in the process of accepting restorative justice as a viable alternative?
+
- Has Impact Justice observed spillover effects from its local restorative justice work?
+
- What progress has been made on national advocacy?
+
- What progress has been made on interactions with the federal government?
+
+
Internal forecasts
+
We’re experimenting with recording explicit numerical forecasts of events related to our decisionmaking (especially grantmaking). The idea behind this is to pull out the implicit predictions that are playing a role in our decisions and make it possible for us to look back on how well-calibrated and accurate those are. For this grant, we are recording the following forecasts:
+
+- 10% chance that we will consider this grant a cost-effective success in one year.
+
- 70% chance that this grant will play an important role in getting traction for the concept of restorative justice on a national level over the next three years.
+
+
Deciding whether to renew the grant
+
By the end of the three-year grant period, we expect to have good information on the success of the grant. It is likely that we will want to renew the grant if:
+
+- Ms. baliga has built a strong platform for restorative justice work and is gaining traction and attention.
+
- Stakeholders in the targeted jurisdictions are signing on to support the restorative justice programs.
+
+
",,"★★★☆☆"
+"Efforts to Improve the Accuracy of Our Judgments and Forecasts","https://www.openphilanthropy.org/blog/efforts-improve-accuracy-our-judgments-and-forecasts","GiveWell",false,"none","
Plans for follow-up
+
Key questions for follow-up
+
Key questions that we intend to ask when following up on 80,000 Hours’ progress include:
+
+- How much progress has 80,000 Hours made on hiring?
+
- How much progress has it made on fundraising?
+
- How many IASPCs does 80,000 Hours believe it has caused since the beginning of the grant period?
+
- Did 80,000 Hours provide plausible evidence that marketing funds were used to increase proxies for outcomes we care about, such as newsletter subscriptions, workshop attendance, or, ideally, plan changes?
+
- Did 80,000 Hours produce written material that we think is high-quality and useful?
+
+
Internal forecasts
+
For this grant, we are recording the following forecasts (though note that these forecasts are rougher than usual):
+
+- 80,000 Hours claims at least 1,500 impact-adjusted plan changes in 2017: 66%
+
- 80,000 Hours claims at least 2,400 impact-adjusted plan changes in 2017: 33%
+
- 80,000 Hours hires at least two staff members in 2017: 60%
+
- 80,000 Hours raises at least $750,000 by February 1, 2017, not including funding from the Open Philanthropy Project: 70%
+
- 80,000 Hours raises at least $1,250,000 by February 1, 2017, not including funding from the Open Philanthropy Project: 35%
+
+
(The above predictions were made on December 16, 2016. In fact, 80,000 Hours raised about $1 million between February 1, 2016 and February 1, 2017.)
+
",,"★★★☆☆"
+"University of Pennsylvania — Philip Tetlock on Forecasting","https://www.openphilanthropy.org/giving/grants/university-pennsylvania-philip-tetlock-forecasting","GiveWell",false,"none","
Plans for follow-up
+
Goals and expectations for this grant
+
We do not have settled expectations for this grant. We expect that there is a large range of possible outcomes that we would be happy with, and that would be hard to predict in advance.
+
We think that it is reasonable to expect that this grant enables Professor Tetlock’s future projects to launch, and perhaps to become more ambitious than they otherwise would have been. We would be pleased if these projects improve our understanding of forecasting and prediction-making in general, or if they provide a platform for Professor Tetlock to bring his existing findings to a more mainstream audience.
+
An example of a very positive outcome, from our perspective, would be if this grant contributes to a shift towards a world in which comparing policy claims to the best available forecasts becomes a standard component of evaluating such claims. We consider it very unlikely that this will be a direct outcome of this grant, but we would be excited if the grant laid some groundwork for further work in this direction.
+
An example of the kind of impact we would consider a moderate success for this type of effort in the longer term is the effect that Nate Silver’s statistical modeling at FiveThirtyEight11 has had on the analysis of presidential elections. We would not characterize this impact as revolutionary, but we believe that it has been notable, and that his analyses and models are well-known and respected among people who think seriously about elections.
+
Internal forecasts
+
We’re experimenting with recording explicit numerical forecasts of events related to our decisionmaking (especially grantmaking), in part in connection with our interest in the topic of this grant. The idea behind this is to pull out the implicit predictions that are playing a role in our decisions, and make it possible for us to look back on how well-calibrated and accurate those are. For this grant, we are recording the following forecast:
+
+- The Alpha Pundit Challenge, or something like it, will have converted five or more vague predictions from pundits into numerical predictions, beyond those described in Tetlock, Alpha Pundit Challenge Proposal, by December 31, 2016: 50%
+
Key questions for follow up
+
Questions we anticipate asking Professor Tetlock as part of our follow-up on this grant include:
+
+- How was the funding spent?
+
- What progress has been made on Adversarial Collaboration Tournaments, the Alpha Pundit Challenge, and any other relevant projects?
+
- Did interesting or high profile people become involved with either project?
+
- Was he able to attract significant public attention to either project?
+
+
Follow up expectations
+
We plan to speak to Professor Tetlock about these projects roughly every six months. We anticipate producing a written update on this grant after one or two years. If we decide at some point to make a follow up grant, we expect that we would produce a written update at that time.
+
",,"★★★☆☆"
\ No newline at end of file
diff --git a/data/givewellopenphil-questions.json b/data/givewellopenphil-questions.json
new file mode 100644
index 0000000..ce95acc
--- /dev/null
+++ b/data/givewellopenphil-questions.json
@@ -0,0 +1,344 @@
+[
+ {
+ "Title": "One for the World — General Support",
+ "URL": "https://www.givewell.org/about/impact/one-for-the-world/july-2018-grant#Internal_forecasts",
+ "Platform": "GiveWell",
+ "Binary question?": false,
+ "Percentage": "none",
+ "Description": "
Internal forecasts
For this grant, we are recording the following forecasts:
| Confidence | Prediction | By Time |
|---|
| 25% | OFTW moves more than $2.5 million to GiveWell top charities in 2020. | End of 2020 |
| 15% | Conditioned on it still being active, OFTW moves more than $5 million to GiveWell top charities in 2023. | End of 2023 |
| 75% | We renew our support to OFTW after one year. | September 2019 |
| 50% | We renew our support to OFTW after two years. | September 2020 |
",
+ "Stars": "★★★☆☆"
+ },
+ {
+ "Title": "Georgetown University Initiative on Innovation, Development, and",
+ "URL": "https://www.givewell.org/charities/gui2de/january-2017-grant#Internal_forecasts",
+ "Platform": "GiveWell",
+ "Binary question?": false,
+ "Percentage": "none",
+ "Description": "
Internal forecasts
We are experimenting with recording explicit numerical forecasts of the probability of events related to our decision-making (especially grant-making). The idea behind this is to pull out the implicit predictions that are playing a role in our decisions, and to make it possible for us to look back on how well-calibrated and accurate those predictions were. For this grant, Josh Rosenberg, our senior research analyst who led GiveWell's investigation of Zusha!, records the following forecasts:
- Zusha! is recommended as a top charity by year-end 2017: 35%, broken down into:
- Zusha! appears more cost-effective than AMF: 10%
- Zusha! appears roughly as cost-effective as AMF: 15%
- Zusha! appears less cost-effective than AMF (but is still a top charity recommendation): 10%
",
+ "Stars": "★★★☆☆"
+ },
+ {
+ "Title": "Charity Science: Health — General Support",
+ "URL": "https://www.givewell.org/charities/charity-science/charity-science-health/november-2016-grant#Internal_forecasts",
+ "Platform": "GiveWell",
+ "Binary question?": false,
+ "Percentage": "none",
+ "Description": "
Internal forecasts
We are experimenting with recording explicit numerical forecasts of the probability of events related to our decision-making (especially grant-making). The idea behind this is to pull out the implicit predictions that are playing a role in our decisions, and to make it possible for us to look back on how well-calibrated and accurate those predictions were. For this grant, we are recording the following forecasts:
- Good Ventures gives additional funding to Charity Science: Health in one year: 80%
- Charity Science: Health becomes (or creates) a GiveWell top charity by giving season 2019: 15%
",
+ "Stars": "★★★☆☆"
+ },
+ {
+ "Title": "Charity Science Health — SMS Reminders for Immunization",
+ "URL": "https://www.givewell.org/charities/charity-science/charity-science-health/july-2017-grant#Internal_forecasts",
+ "Platform": "GiveWell",
+ "Binary question?": false,
+ "Percentage": "none",
+ "Description": "
Internal forecasts
For this grant, we are recording the following forecasts:
- 80% chance that we will recommend another GiveWell Incubation Grant to Charity Science Health by August 2018.
- 15% chance that Charity Science Health will be a GiveWell top charity by the end of 2019.
",
+ "Stars": "★★★☆☆"
+ },
+ {
+ "Title": "Results for Development — Childhood Pneumonia Treatment Scale-Up",
+ "URL": "https://www.givewell.org/charities/results-for-development/may-2016-grant#Internal_forecasts",
+ "Platform": "GiveWell",
+ "Binary question?": false,
+ "Percentage": "none",
+ "Description": "
Internal forecasts
We are experimenting with recording explicit numerical forecasts of the probability of events related to our decision-making (especially grant-making). The idea behind this is to pull out the implicit predictions that are playing a role in our decisions, and to make it possible for us to look back on how well-calibrated and accurate those predictions were. For this grant, we are recording the following forecasts:
- Good Ventures gives R4D a second grant of approximately the same size in 12 months: 70%
- R4D is a top charity by the end of 2019: 25%
",
+ "Stars": "★★★☆☆"
+ },
+ {
+ "Title": "New Incentives — General Support (November 2017)",
+ "URL": "https://www.givewell.org/charities/new-incentives/november-2017-grant#Internal_forecasts",
+ "Platform": "GiveWell",
+ "Binary question?": false,
+ "Percentage": "none",
+ "Description": "
Internal forecasts
We are experimenting with recording explicit numerical forecasts of the probability of events related to our decision-making (especially grant-making). The purpose of this exercise is to record the implicit predictions that inform our decisions and to make it possible for us to look back on how well-calibrated and accurate those predictions were. For this grant, we are recording the forecasts below, all of which we consider to be fairly rough. Except where otherwise noted, the end date for all predictions is the end of 2020.
- New Incentives increases vaccination rates by >17 percentage points and this is detected by the RCT: 15%
- New Incentives increases vaccination rates by >17 percentage points and this is not detected by the RCT: small probability, close to 0%
- New Incentives increases vaccination rates by between 6 and 17 percentage points and this is detected by the RCT: 55%
- New Incentives increases vaccination rates by between 6 and 17 percentage points and this is either not detected by the RCT or is unclear: 15%
- New Incentives increases vaccination rates by <6 percentage points and we either conclude as much or are uncertain enough that we choose not to pursue New Incentives further: 15%
- New Incentives increases vaccination rates by <6 percentage points and we falsely believe it is higher and do pursue New Incentives further: 5%
- After seeing the RCT results, we are significantly uncertain about whether or not to recommend New Incentives as a top charity: 20%
- GiveWell estimates that New Incentives is >3x as cost-effective as GiveDirectly: 50%
- GiveWell estimates that New Incentives is >2x as cost-effective as AMF: <10%
- New Incentives becomes a top charity by November 2020: 50%
",
+ "Stars": "★★★☆☆"
+ },
+ {
+ "Title": "New Incentives — General Support (2016)",
+ "URL": "https://www.givewell.org/charities/new-incentives/march-2016-grant#Internal_forecasts",
+ "Platform": "GiveWell",
+ "Binary question?": false,
+ "Percentage": "none",
+ "Description": "
Internal forecasts
We are experimenting with recording explicit numerical forecasts of the probability of events related to our decision-making (especially grant-making). The idea behind this is to pull out the implicit predictions that are playing a role in our decisions, and to make it possible for us to look back on how well-calibrated and accurate our past predictions were. For this grant, we are recording the following forecasts (made during our decision process):
Top charity predictions
- New Incentives is a top charity in 2016: 10%
- New Incentives is a top charity in 2017: 12.5%
- New Incentives is a top charity in 2018: 15%
Cost-effectiveness predictions
- Our 2017 cost-effectiveness estimate for New Incentives is at least twice as good as our 2017 estimate for unconditional cash transfers: 67%
- Our 2017 cost-effectiveness estimate for New Incentives is at least five times as good as our 2017 estimate for unconditional cash transfers: 15%
- Our 2017 cost-effectiveness estimate for New Incentives is at least ten times as good as our 2017 estimate for unconditional cash transfers: 5%
Charity predictions
- New Incentives brings in at least $250,000 from a funder other than Good Ventures and the Lampert Family Foundation by the end of 2018: 25%
- New Incentives still operates in 2019: 40%
",
+ "Stars": "★★★☆☆"
+ },
+ {
+ "Title": "New Incentives — General Support",
+ "URL": "https://www.givewell.org/charities/new-incentives/april-2017-grant#Internal_forecasts",
+ "Platform": "GiveWell",
+ "Binary question?": false,
+ "Percentage": "none",
+ "Description": "
Internal forecasts
We are experimenting with recording explicit numerical forecasts of the probability of events related to our decision-making (especially grant-making). The purpose of this exercise is to record the implicit predictions that inform our decisions, and to make it possible for us to look back on how well-calibrated and accurate those predictions were. For this grant, we are recording the following forecasts:
- 70% chance that we provide funding for an RCT of New Incentives' program
- 50% chance that New Incentives is a top charity at the end of 2019
",
+ "Stars": "★★★☆☆"
+ },
+ {
+ "Title": "Evidence Action — No Lean Season (December 2016 grant)",
+ "URL": "https://www.givewell.org/charities/evidence-action/december-2016-grant#Internal_forecasts",
+ "Platform": "GiveWell",
+ "Binary question?": false,
+ "Percentage": "none",
+ "Description": "
Internal forecasts
We are experimenting with recording explicit numerical forecasts of the probability of events related to our decision-making (especially grant-making). The idea behind this is to pull out the implicit predictions that are playing a role in our decisions, and to make it possible for us to look back on how well-calibrated and accurate those predictions were. For this grant, we are recording the following forecast:
- 65% chance that No Lean Season is a top charity at the end of giving season 2017
",
+ "Stars": "★★★☆☆"
+ },
+ {
+ "Title": "Evidence Action — Strengthen Operations",
+ "URL": "https://www.givewell.org/charities/evidence-action/april-2017-grant#Internal_forecasts",
+ "Platform": "GiveWell",
+ "Binary question?": false,
+ "Percentage": "none",
+ "Description": "
Internal forecasts
For this grant, we are recording the following forecasts:
- 15% chance that we find a significant error in Evidence Action's financial documents in 2018
- 25% chance that an Evidence Action Beta program other than No Lean Season becomes a top charity by the end of 2021
- 60% chance that the Deworm the World Initiative's room for more funding (including execution levels 1 and 2) exceeds $10 million as of November 2018
- 60% chance that GiveWell Incubation Grants provides at least $250,000 to an Evidence Action Beta program other than No Lean Season by the end of 2018
",
+ "Stars": "★★★☆☆"
+ },
+ {
+ "Title": "Evidence Action — No Lean Season (March 2016 grant)",
+ "URL": "https://www.givewell.org/evidence-action/march-2016-grant#Internal_forecasts",
+ "Platform": "GiveWell",
+ "Binary question?": false,
+ "Percentage": "none",
+ "Description": "
Internal forecasts
We’re experimenting with recording explicit numerical forecasts of events related to our decisionmaking (especially grantmaking). The idea behind this is to pull out the implicit predictions that are playing a role in our decisions, and make it possible for us to look back on how well-calibrated and accurate those are. For this grant, we are recording the following forecasts:
- Top charity predictions:
- No Lean Season (or a related organization) is a top charity in 2017: 15%.
- No Lean Season (or a related organization) is a top charity in 2018: 20%.
- No Lean Season (or a related organization) is a top charity in 2019: 25%.
- Cost-effectiveness predictions:
- Our 2016 cost-effectiveness estimate for No Lean Season is at least five times as good as cash transfers: 50%.
- Our 2016 cost-effectiveness estimate for No Lean Season is less than twice as good as cash transfers: 15%.
- Our 2016 cost-effectiveness estimate for No Lean Season is at least ten times as good as cash transfers: 15%.
- Implementation predictions:
- Evidence Action is running a No Lean Season program at the end of 2017: 80%.
",
+ "Stars": "★★★☆☆"
+ },
+ {
+ "Title": "Innovations for Poverty Action — Mindset Engagement in Cash Transfers",
+ "URL": "https://www.givewell.org/international/charities/ipa/may-2016-grant#Risks_of_the_grant_and_internal_forecasts",
+ "Platform": "GiveWell",
+ "Binary question?": false,
+ "Percentage": "none",
+ "Description": "
Risks of the grant and internal forecasts
This grant could fail to have the effects we hope for in a number of ways:
- The study detects an effect that is too small relative to the cost of implementing the intervention for it to be worth scaling up. We believe this is reasonably likely (~50% chance).
- The study yields a result that we're not confident in. We think there is a moderate chance (~25%) of this (given the number of potential problems that can arise with any study).
- The study detects an effect that would be worth scaling up, but we are unable to find an implementer interested in doing so (for instance, if GiveDirectly were to decide not to incorporate the intervention because it is too time-intensive or diverts attention from other activities, or because GiveDirectly interprets the study's results differently than we do). We think this scenario is fairly unlikely (~7.5%).
- The intervention has no measurable effect, and we could have predicted this prior to the study by surveying the existing literature more thoroughly. We think this is fairly unlikely (~7.5%), especially given Sedlmayr's interest in attempting the intervention.
(We’re experimenting with recording explicit numerical forecasts of events related to our decisionmaking, especially grantmaking. The idea behind this is to pull out the implicit predictions that are playing a role in our decisions, and make it possible for us to look back on how well-calibrated and accurate those are.)
",
+ "Stars": "★★★☆☆"
+ },
+ {
+ "Title": "Evidence Action Beta — Iron and Folic Acid Supplementation ("Phase 2")",
+ "URL": "https://www.givewell.org/research/incubation-grants/december-2018-evidence-action-beta-iron-folic-acid-phase-2#Internal_forecasts",
+ "Platform": "GiveWell",
+ "Binary question?": false,
+ "Percentage": "none",
+ "Description": "
Internal forecasts
For this grant, we are recording the following forecasts:
| Confidence | Prediction | By Time |
|---|
| 60% | GiveWell’s best guess is that Evidence Action’s intervention increases coverage relative to the counterfactual in the first year of Phase 2 of the program by at least 4 percentage points | December 2021 |
| 50% | GiveWell’s best guess is that Evidence Action’s intervention increases coverage relative to the counterfactual in the second year of Phase 2 of the program by at least 8 percentage points (cumulatively) | December 2021 |
| 75% | Evidence Action requests funding for Phase 3 of this program because it believes Phase 2 to have been successful | December 2021 |
| 80% | Estimates of anemia rates from the India National Family Health Survey in an average of 5 randomly chosen non-Evidence Action-supported states do not show anemia declining by more than 2 percentage points per year over the last 5 years (e.g., due to iron fortification or other changes) | January 2024 |
| 35% | Evidence Action ultimately spends at least $15 million total on IFA technical assistance that we retrospectively model as 10x as effective (or more) than cash transfers (using our January 2018 CEA as a baseline) | January 2025 |
",
+ "Stars": "★★★☆☆"
+ },
+ {
+ "Title": "UC Berkeley — KLPS-4 Survey",
+ "URL": "https://www.givewell.org/research/incubation-grants/uc-berkeley/april-2017-grant#Internal_forecasts",
+ "Platform": "GiveWell",
+ "Binary question?": false,
+ "Percentage": "none",
+ "Description": "
Plans for follow-up
We plan to follow up with the gift recipient roughly every six months to check in on the timeline for receiving results from this study. At this stage, our understanding is that Wave 1 results will be available by mid-2018 and Wave 2 results will be available by mid-2019. We are uncertain when results will be able to be shared publicly, but aim to write publicly about the results as soon as we are able to.
We also plan to follow up with the recipient to share their pre-analysis plan publicly and, when the study is completed, to share data publicly.
Internal forecasts
We’re experimenting with recording explicit numerical forecasts of events related to our decisionmaking (especially grantmaking). The idea behind this is to pull out the implicit predictions that are playing a role in our decisions, and make it possible for us to look back on how well-calibrated and accurate those are. For this gift, we are recording the following forecasts:
- 25% chance that the KLPS-4 survey significantly positively updates us on deworming, i.e. finds a result that increases our estimated cost-effectiveness for deworming by at least 2x.
- 5% chance that the KLPS-4 survey significantly negatively updates us on deworming, i.e. finds a result that decreases our estimated cost-effectiveness for deworming by at least 2x.