feat: big update
This commit is contained in:
parent
27e81ede93
commit
1e9f35fda0
2
.blog/_werc/config
Executable file
2
.blog/_werc/config
Executable file
|
|
@ -0,0 +1,2 @@
|
||||||
|
# conf_enable_comments -n
|
||||||
|
conf_enable_blog
|
||||||
11
_werc/lib/footer.inc
Executable file
11
_werc/lib/footer.inc
Executable file
|
|
@ -0,0 +1,11 @@
|
||||||
|
<br class="doNotDisplay doNotPrint" />
|
||||||
|
|
||||||
|
<div style="margin-right: auto;">Powered by <a href="http://werc.cat-v.org/">werc</a>, <a href="https://alpinelinux.org/">Alpine Linux</a> and <a href="https://nginx.org/en/">nginx</a></div>
|
||||||
|
|
||||||
|
<!-- TODO: wait until duckduckgo indexes site
|
||||||
|
<form action="https://duckduckgo.com/" method="get">
|
||||||
|
<input type="hidden" name="sites" value="nunosempere.com">
|
||||||
|
<input type="search" name="q">
|
||||||
|
<input type="submit" value="Search">
|
||||||
|
</form>
|
||||||
|
-->
|
||||||
3
index.md
3
index.md
|
|
@ -10,4 +10,5 @@ Scott Alexander [describes us as](https://astralcodexten.substack.com/p/mantic-m
|
||||||
|
|
||||||
> Enter Samotsvety Forecasts. This is a team of some of the best superforecasters in the world. They won the CSET-Foretell forecasting competition by an absolutely obscene margin, “around twice as good as the next-best team in terms of the relative Brier score”. If the point of forecasting tournaments is to figure out who you can trust, the science has spoken, and the answer is “these guys”.
|
> Enter Samotsvety Forecasts. This is a team of some of the best superforecasters in the world. They won the CSET-Foretell forecasting competition by an absolutely obscene margin, “around twice as good as the next-best team in terms of the relative Brier score”. If the point of forecasting tournaments is to figure out who you can trust, the science has spoken, and the answer is “these guys”.
|
||||||
|
|
||||||
We are open to forecasting consulting requests, and can be reached out at [info@samotsvety.org](mailto:info@samotsvety.org).
|
We are open to forecasting consulting requests, and can be reached out at [info@samotsvety.org](mailto:info@samotsvety.org). Readers might also want to view our [track record](./track-record), browse our [projects](./projects) or read our [media mentions](./media-mentions).
|
||||||
|
|
||||||
|
|
|
||||||
23
media-mentions/index.md
Normal file
23
media-mentions/index.md
Normal file
|
|
@ -0,0 +1,23 @@
|
||||||
|
## Media mentions
|
||||||
|
|
||||||
|
N.B.: We are open to media mentions or collaborations, and can be reached at [info@samotsvety.org](mailto:info@samotsvety.org).
|
||||||
|
|
||||||
|
[CNN: How to assess the risk of nuclear war without freaking out](https://edition.cnn.com/2022/06/28/opinions/nuclear-war-likelihood-probability-russia-us-scoblic-mandel/index.html) ([a](https://web.archive.org/web/20220628081603/https://edition.cnn.com/2022/06/28/opinions/nuclear-war-likelihood-probability-russia-us-scoblic-mandel/index.html)):
|
||||||
|
|
||||||
|
> One group of [highly regarded forecasters](https://forum.effectivealtruism.org/posts/KRFXjCqqfGQAYirm5/samotsvety-nuclear-risk-forecasts-march-2022#_blank) put the probability of Russia using a nuclear weapon against London before February 2023 at 0.8%[^1]
|
||||||
|
|
||||||
|
[El País: Is it possible to predict the future of the war in Ukraine? Online forecasting communities think so](Is it possible to predict the future of the war in Ukraine? Online forecasting communities think so) ([a](https://web.archive.org/web/20220401061002/https://english.elpais.com/science-tech/2022-03-26/is-it-possible-to-predict-the-future-of-the-war-in-ukraine-online-forecasting-communities-think-so.html), [es](https://elpais.com/tecnologia/2022-03-24/se-puede-predecir-el-futuro-de-la-guerra-en-ucrania-es-lo-que-intenta-una-comunidad-de-pronosticadores-en-internet.html)):
|
||||||
|
|
||||||
|
> What risk is there for an alert citizen in London of dying next month due to a nuclear explosion? Some 24 micro deaths, 24 options in a million, according to the group of renowned forecasters of which Nuño Sempere, from Madrid, is a part.
|
||||||
|
|
||||||
|
[Astral Codex Ten: Mantic Monday](https://astralcodexten.substack.com/p/mantic-monday-31422) ([a](https://web.archive.org/web/20220401013834/https://astralcodexten.substack.com/p/mantic-monday-31422)):
|
||||||
|
|
||||||
|
> Enter Samotsvety Forecasts. This is a team of some of the best superforecasters in the world. They won the CSET-Foretell forecasting competition by [an absolutely obscene margin](https://www.lesswrong.com/posts/EGHtomuh55375u7RT/forecasting-newsletter-march-2021), “around twice as good as the next-best team in terms of the relative Brier score”. If the point of forecasting tournaments is to figure out who you can trust, the science has spoken, and the answer is “these guys”.
|
||||||
|
|
||||||
|
[INFER: The Making of a Top Forecaster: Techniques to Boost Accuracy](https://www.infer-pub.com/blog/top-forecaster-techniques) ([a](https://web.archive.org/web/20220221032529/https://www.infer-pub.com/blog/top-forecaster-techniques)):
|
||||||
|
|
||||||
|
> The Good Judgment Project’s research also showed the value of working in teams. Teams who debated their forecasts (usually virtually) performed better than individuals working alone. Perhaps that’s yet another technique benefitting top forecaster Sempere (@Loki), who meets regularly with his two teammates, @yagudin (also in the top 1%) and @elifland, to discuss and evaluate questions from different angles.
|
||||||
|
|
||||||
|
<br>
|
||||||
|
|
||||||
|
[^1]: We gave a 0.8% _annualized_ probability to a Russia/NATO nuclear exchange killing at least one person in the first month of the conflict.
|
||||||
19
projects/index.md
Normal file
19
projects/index.md
Normal file
|
|
@ -0,0 +1,19 @@
|
||||||
|
## Done as _Samotsvety_:
|
||||||
|
|
||||||
|
- [Samotsvety Nuclear Risk Forecasts — March 2022](https://forum.effectivealtruism.org/posts/KRFXjCqqfGQAYirm5/samotsvety-nuclear-risk-forecasts-march-2022)
|
||||||
|
- [Samotsvety's AI risk forecasts](https://forum.effectivealtruism.org/posts/EG9xDM8YRz4JN4wMN/samotsvety-s-ai-risk-forecasts)
|
||||||
|
- [Prediction Markets in the Corporate Setting](https://forum.effectivealtruism.org/posts/dQhjwHA7LhfE8YpYF/prediction-markets-in-the-corporate-setting)
|
||||||
|
- [GJO Calibration app](https://share.streamlit.io/yagudin/gjo-calibration/main/strmlt.py)
|
||||||
|
- [Micro-grants for forecasting](https://forum.effectivealtruism.org/posts/oqFa8obfyEmvD79Jn/we-are-giving-usd10k-as-forecasting-micro-grants)
|
||||||
|
- [Bottlenecks to more impactful crowd forecasting](https://forum.effectivealtruism.org/posts/S2vfrZsFHn7Wy4ocm/bottlenecks-to-more-impactful-crowd-forecasting-2)
|
||||||
|
|
||||||
|
## With other hats
|
||||||
|
|
||||||
|
- [Forecasting Newsletter](https://forecasting.substack.com/), by Nuño
|
||||||
|
- [Metaforecast](https://metaforecast.org/), by [QURI](https://quantifieduncertainty.org/)
|
||||||
|
- [Squiggle](https://www.squiggle-language.com/), by [QURI](https://quantifieduncertainty.org/)
|
||||||
|
- [Impactful Forecasting Prize](https://forum.effectivealtruism.org/posts/HDoMrQFG76QtkdrZJ/impactful-forecasting-prize-for-forecast-writeups-on-curated), by [Sage](https://forum.effectivealtruism.org/users/sage)
|
||||||
|
- [Pastcasting](http://pastcasting.com/), by [Sage](https://forum.effectivealtruism.org/users/sage)
|
||||||
|
- [Better scoring rules](https://github.com/SamotsvetyForecasting/optimal-scoring), by Nuño
|
||||||
|
- [Alignment Problems With Current Forecasting Platforms](https://arxiv.org/abs/2106.11248), by Nuño and Alex Lawsen
|
||||||
|
- [Comparing top forecasters and domain experts](https://forum.effectivealtruism.org/posts/qZqvBLvR5hX9sEkjR/comparing-top-forecasters-and-domain-experts), by [Arb](https://arbresearch.com/)
|
||||||
BIN
sitemap.gz
BIN
sitemap.gz
Binary file not shown.
4
sitemap.txt
Executable file → Normal file
4
sitemap.txt
Executable file → Normal file
|
|
@ -1 +1,3 @@
|
||||||
https://samotsvety.org:443/test/
|
https://samotsvety.org/blog/
|
||||||
|
https://samotsvety.org/media-mentions/
|
||||||
|
https://samotsvety.org/track-record/
|
||||||
|
|
|
||||||
129
track-record/index.md
Normal file
129
track-record/index.md
Normal file
|
|
@ -0,0 +1,129 @@
|
||||||
|
## Team achievements
|
||||||
|
|
||||||
|
The Samotsvety Forecasting team at INFER/CSET-Foretell—composed out of Nuño, Misha, Eli—took:
|
||||||
|
|
||||||
|
- [1st place in 2020](https://www.infer-pub.com/leaderboards/teams?seasonid=1), with a relative score of -0.912 (vs. -0.062 by the 2nd best team). Individually we finished [5th, 6th, and 7th](https://www.infer-pub.com/leaderboards/seasons/1).
|
||||||
|
- [1st place in 2021](https://www.infer-pub.com/leaderboards/teams?seasonid=2) with a relative score of -2.43 (vs. -1.039 by the 2nd best team and vs. 0.064 by “2021 Pro Forecasters”). Individually we finished [1st, 3rd, and 4th](https://www.infer-pub.com/leaderboards/seasons/2).
|
||||||
|
- We still hold [1st place in 2022](https://www.infer-pub.com/leaderboards/teams?seasonid=3), despite us reducing our participation.
|
||||||
|
|
||||||
|
The above links require signing in, so here is a screenshot of our performance in the 2021 season:
|
||||||
|
|
||||||
|
<figure >
|
||||||
|
|
||||||
|
<br>
|
||||||
|
<figcaption>Note that we are a bit more than twice as good as the next best team, in terms of the relative Brier score</figcaption>
|
||||||
|
</figure>
|
||||||
|
|
||||||
|
As of 2022-09-15, we are also 4th on the nascent [Insight Prediction leaderboard](https://insightprediction.com/leaderboards) as a result of one (1) large bet when we correctly foresaw the Russian invasion of Ukraine and put our money where our mouth was:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## Individuals
|
||||||
|
|
||||||
|
Individually, Samotsvety members are, as of 2022-09-15, occupying all top 4 spots on INFER's [overall leaderboard](https://www.infer-pub.com/leaderboards/overall):
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
All scores mentioned below are as of 2022-09-15:
|
||||||
|
|
||||||
|
#### Misha Yagudin
|
||||||
|
- Runs a research consultancy [Arb Research](https://arbresearch.com/)
|
||||||
|
- [GJOpen](https://www.gjopen.com/forecaster/yagudin) Brier score 0.191 vs. 0.283 median, ratio 0.67
|
||||||
|
- 7th and 3rd place forecaster in the first two seasons of CSET-Foretell/INFER ([I](https://www.infer-pub.com/leaderboards/seasons/1), [II](https://www.infer-pub.com/leaderboards/seasons/2)); [3rd](https://www.infer-pub.com/forecaster/yagudin/performance?alltime=true) all-time (as of 2022/9/15).
|
||||||
|
|
||||||
|
#### Nuño Sempere
|
||||||
|
|
||||||
|
- Researcher at the [Quantified Uncertainty Research Institute](https://quantifieduncertainty.org/)
|
||||||
|
- [GJOpen](https://www.gjopen.com/forecaster/LokiOdinevich) Brier score 0.209 vs. 0.294 median, ratio 0.71
|
||||||
|
- Top 5 forecaster in the first two seasons of CSET-Foretell/INFER ([I](https://www.infer-pub.com/leaderboards/seasons/1),[II](https://www.infer-pub.com/leaderboards/seasons/2)); [4th](https://www.infer-pub.com/forecaster/loki/performance?alltime=true) all-time (as of 2022/9/15)
|
||||||
|
|
||||||
|
#### Alex Lyzhov
|
||||||
|
|
||||||
|
- ML PhD student at NYU
|
||||||
|
- Participates primarily in AI-related forecasts due to expertise
|
||||||
|
- Active Metaculus [contributor](https://www.metaculus.com/accounts/profile/103447/)
|
||||||
|
|
||||||
|
#### Eli Lifland
|
||||||
|
- Figuring out what's up with AI alignment (and sometimes other stuff), writing [Foxy Scout](https://www.foxy-scout.com/).
|
||||||
|
- [GJOpen](https://www.gjopen.com/forecaster/elifland) Brier score 0.23 vs. 0.301 median, ratio 0.76
|
||||||
|
- [1st](https://www.infer-pub.com/forecaster/elifland/performance?alltime=true) forecaster of all time on CSET-Foretell/INFER (as of 2022/9/15)
|
||||||
|
- 2nd in Metaculus [Economist 2021 tournament](https://www.metaculus.com/questions/11438/economist-2021-tournament-winners/), 1st in [Salk Tournament](https://pandemic.metaculus.com/contests/?selected=salk-tournament) (as of 2022/9/10).
|
||||||
|
- Track record is described in detail [here](https://www.lesswrong.com/posts/XAkhqkNQByEaT8MED/personal-forecasting-retrospective-2020-2022)
|
||||||
|
|
||||||
|
#### Jonathan Mann
|
||||||
|
|
||||||
|
- Professional in the financial sector
|
||||||
|
- [GJOpen](https://www.gjopen.com/forecaster/JonathanMann) Brier score 0.155 vs. 0.247 median, ratio 0.63
|
||||||
|
- Pro forecaster on [INFER](https://www.infer-pub.com/forecaster/JonathanMann/performance)
|
||||||
|
- [About + Github](https://jonathanmann.github.io/about/)
|
||||||
|
|
||||||
|
#### Juan Cambeiro
|
||||||
|
|
||||||
|
- MPH student in epidemiology at Columbia University
|
||||||
|
- Biosecurity fellow at the Institute for Progress
|
||||||
|
- [Good Judgment](https://www.gjopen.com/) Superforecaster
|
||||||
|
- Analyst for [Metaculus](https://www.metaculus.com/questions/)
|
||||||
|
- [GJOpen](https://www.gjopen.com/forecaster/juancambeiro) Brier score 0.25 vs. 0.317 median, ratio 0.79
|
||||||
|
- First in [IARPA's FOCUS](https://www.iarpa.gov/research-programs/focus-2) Tournament on COVID-19 forecasting
|
||||||
|
- First in [Good Judgment Open's US Election 2020 challenge](https://www.gjopen.com/challenges/42-us-election-2020/scores)
|
||||||
|
- Second in [Good Judgment Open's In the News 2020 challenge](https://www.gjopen.com/challenges/36-in-the-news-2020/scores)
|
||||||
|
- First in the time active on [Good Judgment Open's Coronavirus Outbreak challenge](https://www.gjopen.com/challenges/43-coronavirus-outbreak/scores), now third
|
||||||
|
|
||||||
|
#### Sam Glover
|
||||||
|
|
||||||
|
- Writes about social science at [samstack.io](https://www.samstack.io)
|
||||||
|
- [Good Judgment](https://www.gjopen.com/) Superforecaster
|
||||||
|
- GJOpen [Brier score](https://www.gjopen.com/forecaster/samglover97) 0.228vs 0.309 median, ratio 0.74, increased significantly since becoming inactive in late 2021
|
||||||
|
|
||||||
|
#### Molly Hickman
|
||||||
|
|
||||||
|
- Data scientist at [nLine, Inc.](https://nline.io/), and previously worked on crowd-sourced prediction projects on the evaluation side.
|
||||||
|
- [Top 2](https://www.infer-pub.com/forecaster/mollygh) forecaster of all time on [INFER](https://www.infer-pub.com/) (as of 2022/9/15); [2nd place forecaster in season 2021](https://www.infer-pub.com/leaderboards/seasons/2) (-1.684 relative Brier score); [8th place forecaster in season 2020](https://www.infer-pub.com/leaderboards/seasons/1) (-0.529 relative Brier score)
|
||||||
|
|
||||||
|
#### Gregory Lewis
|
||||||
|
|
||||||
|
- Ex public health doctor, now working in biosecurity
|
||||||
|
- [Good Judgment](https://www.gjopen.com/) Superforecaster (since 2021)
|
||||||
|
- [GJOpen](https://www.gjopen.com/forecaster/Gjlewis) Brier score 0.226 vs. 0.313 median, ratio 0.72
|
||||||
|
|
||||||
|
|
||||||
|
#### Pablo Stafforini
|
||||||
|
|
||||||
|
- Philosopher by training, currently editor of the *Global Priorities Encyclopedia*.
|
||||||
|
- 1st place in Road to Recovery tournament
|
||||||
|
- 3rd place in Li Wenliang Forecasting Tournament I tournament
|
||||||
|
- 1st place in Li Wenliang Forecasting Tournament II tournament
|
||||||
|
- See [my Metaculus profile](https://www.metaculus.com/accounts/profile/101341/) for additional information.
|
||||||
|
|
||||||
|
#### Jared Leibowich
|
||||||
|
|
||||||
|
- Data Scientist
|
||||||
|
- Forecaster for the [Swift Centre](https://www.swiftcentre.org/)
|
||||||
|
- [GJO](https://www.gjopen.com/forecaster/jleibowich) Brier score 0.22 vs. 0.303 median, ratio 0.73
|
||||||
|
- Placed 1st out of 7,000 ["In the News 2021" Good Judgment competition](https://www.gjopen.com/leaderboards/challenges/47-in-the-news-2021)
|
||||||
|
- Currently ranked 1st out of 3,000 ["In the News 2022" Good Judgment competition](https://www.gjopen.com/leaderboards/challenges/57-in-the-news-2022)
|
||||||
|
|
||||||
|
#### Greg Justice
|
||||||
|
|
||||||
|
- Analyst in the healthcare space
|
||||||
|
- [GJOpen](https://www.gjopen.com/forecaster/slapthepancake) Brier score 0.187 vs. 0.251 median, ratio 0.75
|
||||||
|
- 1st place in [Adam Grant's Think Again challenge on GJ Open](https://adamgrant.bulletin.com/how-to-be-less-wrong)
|
||||||
|
(as of 2022-09-11)
|
||||||
|
- 4th place - Olivier Sibony's Noise challenge on GJ Open (as of 2022-09-11)
|
||||||
|
- 7th place - Good Judgment Project 2.0, in initial accuracy (as of 2022-09-11)
|
||||||
|
|
||||||
|
#### @belikewater
|
||||||
|
|
||||||
|
- MA in Economics, PhD in Neuroscience
|
||||||
|
- [GJOpen](https://www.gjopen.com/forecaster/belikewater) Brier score 0.215 vs. 0.279 median, ratio 0.77
|
||||||
|
- Recently, also forecasting at INFER, Metaculus, Manifold Markets
|
||||||
|
|
||||||
|
#### Tolga Bilge
|
||||||
|
|
||||||
|
- Mathematics student
|
||||||
|
- GJI Superforecaster
|
||||||
|
- Forecaster for the [Swift Centre](https://swiftcentre.org)
|
||||||
|
- [GJOpen](https://www.gjopen.com/forecaster/Tolga) Brier score 0.164 vs. 0.254 median, ratio 0.65
|
||||||
|
- Forecaster on [INFER](https://www.infer-pub.com/forecaster/Tolga)
|
||||||
|
|
||||||
|
[^1]: Says Pablo.
|
||||||
Loading…
Reference in New Issue
Block a user