samotsvety.org/track-record/index.md

138 lines
8.2 KiB
Markdown
Raw Normal View History

2022-09-18 11:16:24 +00:00
## Team achievements
The Samotsvety Forecasting team at INFER/CSET-Foretell—composed out of Nuño, Misha, Eli—took:
- [1st place in 2020](https://www.infer-pub.com/leaderboards/teams?seasonid=1), with a relative score of -0.912 (vs. -0.062 by the 2nd best team). Individually we finished [5th, 6th, and 7th](https://www.infer-pub.com/leaderboards/seasons/1).
- [1st place in 2021](https://www.infer-pub.com/leaderboards/teams?seasonid=2) with a relative score of -2.43 (vs. -1.039 by the 2nd best team and vs. 0.064 by “2021 Pro Forecasters”). Individually we finished [1st, 3rd, and 4th](https://www.infer-pub.com/leaderboards/seasons/2).
- We still hold [1st place in 2022](https://www.infer-pub.com/leaderboards/teams?seasonid=3), despite us reducing our participation.
The above links require signing in, so here is a screenshot of our performance in the 2021 season:
<figure >
![](https://i.imgur.com/4hayqba.png)
<br>
<figcaption>Note that we are a bit more than twice as good as the next best team, in terms of the relative Brier score</figcaption>
</figure>
As of 2022-09-15, we are also 4th on the nascent [Insight Prediction leaderboard](https://insightprediction.com/leaderboards) as a result of one (1) large bet when we correctly foresaw the Russian invasion of Ukraine and put our money where our mouth was:
![](https://i.imgur.com/ob5Cla4.png)
## Individuals
Individually, Samotsvety members are, as of 2022-09-15, occupying all top 4 spots on INFER's [overall leaderboard](https://www.infer-pub.com/leaderboards/overall):
![](https://i.imgur.com/ygMxoht.png)
For each forecaster below, the time at which their profile was last updated is indicated in a superscript.
2022-09-18 11:16:24 +00:00
#### Misha Yagudin<sup>2022-09-15</sup>
2022-09-18 11:16:24 +00:00
- Runs a research consultancy [Arb Research](https://arbresearch.com/)
- [GJOpen](https://www.gjopen.com/forecaster/yagudin) Brier score 0.191 vs. 0.283 median, ratio 0.67
- 7th and 3rd place forecaster in the first two seasons of CSET-Foretell/INFER ([I](https://www.infer-pub.com/leaderboards/seasons/1), [II](https://www.infer-pub.com/leaderboards/seasons/2)); [3rd](https://www.infer-pub.com/forecaster/yagudin/performance?alltime=true) all-time (as of 2022/9/15).
#### Nuño Sempere<sup>2022-09-15</sup>
2022-09-18 11:16:24 +00:00
- Researcher at the [Quantified Uncertainty Research Institute](https://quantifieduncertainty.org/)
- [GJOpen](https://www.gjopen.com/forecaster/LokiOdinevich) Brier score 0.209 vs. 0.294 median, ratio 0.71
- Top 5 forecaster in the first two seasons of CSET-Foretell/INFER ([I](https://www.infer-pub.com/leaderboards/seasons/1),[II](https://www.infer-pub.com/leaderboards/seasons/2)); [4th](https://www.infer-pub.com/forecaster/loki/performance?alltime=true) all-time (as of 2022/9/15)
#### Alex Lyzhov<sup>2022-09-15</sup>
2022-09-18 11:16:24 +00:00
- ML PhD student at NYU
- Participates primarily in AI-related forecasts due to expertise
- Active Metaculus [contributor](https://www.metaculus.com/accounts/profile/103447/)
#### Eli Lifland<sup>2022-09-15</sup>
2022-09-18 11:16:24 +00:00
- Figuring out what's up with AI alignment (and sometimes other stuff), writing [Foxy Scout](https://www.foxy-scout.com/).
- [GJOpen](https://www.gjopen.com/forecaster/elifland) Brier score 0.23 vs. 0.301 median, ratio 0.76
- [1st](https://www.infer-pub.com/forecaster/elifland/performance?alltime=true) forecaster of all time on CSET-Foretell/INFER (as of 2022/9/15)
- 2nd in Metaculus [Economist 2021 tournament](https://www.metaculus.com/questions/11438/economist-2021-tournament-winners/), 1st in [Salk Tournament](https://pandemic.metaculus.com/contests/?selected=salk-tournament) (as of 2022/9/10).
- Track record is described in detail [here](https://www.lesswrong.com/posts/XAkhqkNQByEaT8MED/personal-forecasting-retrospective-2020-2022)
#### Jonathan Mann<sup>2022-09-15</sup>
2022-09-18 11:16:24 +00:00
- Professional in the financial sector
- [GJOpen](https://www.gjopen.com/forecaster/JonathanMann) Brier score 0.155 vs. 0.247 median, ratio 0.63
- Pro forecaster on [INFER](https://www.infer-pub.com/forecaster/JonathanMann/performance)
- [About + Github](https://jonathanmann.github.io/about/)
#### Juan Cambeiro<sup>2022-09-15</sup>
2022-09-18 11:16:24 +00:00
- MPH student in epidemiology at Columbia University
- Biosecurity fellow at the Institute for Progress
- [Good Judgment](https://www.gjopen.com/) Superforecaster
- Analyst for [Metaculus](https://www.metaculus.com/questions/)
- [GJOpen](https://www.gjopen.com/forecaster/juancambeiro) Brier score 0.25 vs. 0.317 median, ratio 0.79
- First in [IARPA's FOCUS](https://www.iarpa.gov/research-programs/focus-2) Tournament on COVID-19 forecasting
- First in [Good Judgment Open's US Election 2020 challenge](https://www.gjopen.com/challenges/42-us-election-2020/scores)
- Second in [Good Judgment Open's In the News 2020 challenge](https://www.gjopen.com/challenges/36-in-the-news-2020/scores)
- First in the time active on [Good Judgment Open's Coronavirus Outbreak challenge](https://www.gjopen.com/challenges/43-coronavirus-outbreak/scores), now third
#### Sam Glover<sup>2022-09-15</sup>
2022-09-18 11:16:24 +00:00
- Writes about social science at [samstack.io](https://www.samstack.io)
- [Good Judgment](https://www.gjopen.com/) Superforecaster
- GJOpen [Brier score](https://www.gjopen.com/forecaster/samglover97) 0.228vs 0.309 median, ratio 0.74, increased significantly since becoming inactive in late 2021
#### Molly Hickman<sup>2022-09-15</sup>
2022-09-18 11:16:24 +00:00
- Data scientist at [nLine, Inc.](https://nline.io/), and previously worked on crowd-sourced prediction projects on the evaluation side.
- [Top 2](https://www.infer-pub.com/forecaster/mollygh) forecaster of all time on [INFER](https://www.infer-pub.com/) (as of 2022/9/15); [2nd place forecaster in season 2021](https://www.infer-pub.com/leaderboards/seasons/2) (-1.684 relative Brier score); [8th place forecaster in season 2020](https://www.infer-pub.com/leaderboards/seasons/1) (-0.529 relative Brier score)
#### Gregory Lewis<sup>2022-09-15</sup>
2022-09-18 11:16:24 +00:00
- Ex public health doctor, now working in biosecurity
- [Good Judgment](https://www.gjopen.com/) Superforecaster (since 2021)
- [GJOpen](https://www.gjopen.com/forecaster/Gjlewis) Brier score 0.226 vs. 0.313 median, ratio 0.72
#### Pablo Stafforini<sup>2022-09-15</sup>
2022-09-18 11:16:24 +00:00
- Philosopher by training, currently editor of the *Global Priorities Encyclopedia*.
- 1st place in Road to Recovery tournament
- 3rd place in Li Wenliang Forecasting Tournament I tournament
- 1st place in Li Wenliang Forecasting Tournament II tournament
- See [my Metaculus profile](https://www.metaculus.com/accounts/profile/101341/) for additional information.
#### Jared Leibowich<sup>2022-09-15</sup>
2022-09-18 11:16:24 +00:00
- Data Scientist
- Forecaster for the [Swift Centre](https://www.swiftcentre.org/)
- [GJO](https://www.gjopen.com/forecaster/jleibowich) Brier score 0.22 vs. 0.303 median, ratio 0.73
- Placed 1st out of 7,000 ["In the News 2021" Good Judgment competition](https://www.gjopen.com/leaderboards/challenges/47-in-the-news-2021)
- Currently ranked 1st out of 3,000 ["In the News 2022" Good Judgment competition](https://www.gjopen.com/leaderboards/challenges/57-in-the-news-2022)
#### Greg Justice<sup>2022-09-15</sup>
2022-09-18 11:16:24 +00:00
- Analyst in the healthcare space
- [GJOpen](https://www.gjopen.com/forecaster/slapthepancake) Brier score 0.187 vs. 0.251 median, ratio 0.75
- 1st place in [Adam Grant's Think Again challenge on GJ Open](https://adamgrant.bulletin.com/how-to-be-less-wrong)
(as of 2022-09-11)
- 4th place - Olivier Sibony's Noise challenge on GJ Open (as of 2022-09-11)
- 7th place - Good Judgment Project 2.0, in initial accuracy (as of 2022-09-11)
#### @belikewater<sup>2022-09-15</sup>
2022-09-18 11:16:24 +00:00
- MA in Economics, PhD in Neuroscience
- [GJOpen](https://www.gjopen.com/forecaster/belikewater) Brier score 0.215 vs. 0.279 median, ratio 0.77
- Recently, also forecasting at INFER, Metaculus, Manifold Markets
#### Tolga Bilge<sup>2022-09-15</sup>
2022-09-18 11:16:24 +00:00
- Mathematics student
- GJI Superforecaster
- Forecaster for the [Swift Centre](https://swiftcentre.org)
- [GJOpen](https://www.gjopen.com/forecaster/Tolga) Brier score 0.164 vs. 0.254 median, ratio 0.65
- Forecaster on [INFER](https://www.infer-pub.com/forecaster/Tolga)
#### Vidur Kapur<sup>2032-01-22</sup>
- Holds degrees in Medicine and Public Health
- Good Judgment Superforecaster
- GJ Open Brier score 0.276 vs. 0.372 median, ratio 0.74
- 4th place forecaster for both initial and final forecast in the Good Judgment Project 2.0 (GJP 2.0) COVID-19 forecasting tournament
2022-09-18 11:16:24 +00:00
[^1]: Says Pablo.