samotsvety.org/track-record/index.md

Team achievements

The Samotsvety Forecasting team at INFER/CSET-Foretell—composed out of Nuño, Misha, Eli—took:

• 1st place in 2020, with a relative score of -0.912 (vs. -0.062 by the 2nd best team). Individually we finished 5th, 6th, and 7th.
• 1st place in 2021 with a relative score of -2.43 (vs. -1.039 by the 2nd best team and vs. 0.064 by “2021 Pro Forecasters”). Individually we finished 1st, 3rd, and 4th.
• We still hold 1st place in 2022, despite us reducing our participation.

The above links require signing in, so here is a screenshot of our performance in the 2021 season:


Note that we are a bit more than twice as good as the next best team, in terms of the relative Brier score

As of 2022-09-15, we are also 4th on the nascent Insight Prediction leaderboard as a result of one (1) large bet when we correctly foresaw the Russian invasion of Ukraine and put our money where our mouth was:

Individuals

Individually, Samotsvety members are, as of 2022-09-15, occupying all top 4 spots on INFER's overall leaderboard:

All scores mentioned below are as of 2022-09-15:

Misha Yagudin

• Runs a research consultancy Arb Research
• GJOpen Brier score 0.191 vs. 0.283 median, ratio 0.67
• 7th and 3rd place forecaster in the first two seasons of CSET-Foretell/INFER (I, II); 3rd all-time (as of 2022/9/15).

Nuño Sempere

• Researcher at the Quantified Uncertainty Research Institute
• GJOpen Brier score 0.209 vs. 0.294 median, ratio 0.71
• Top 5 forecaster in the first two seasons of CSET-Foretell/INFER (I,II); 4th all-time (as of 2022/9/15)

Alex Lyzhov

• ML PhD student at NYU
• Participates primarily in AI-related forecasts due to expertise
• Active Metaculus contributor

Eli Lifland

• Figuring out what's up with AI alignment (and sometimes other stuff), writing Foxy Scout.
• GJOpen Brier score 0.23 vs. 0.301 median, ratio 0.76
• 1st forecaster of all time on CSET-Foretell/INFER (as of 2022/9/15)
• 2nd in Metaculus Economist 2021 tournament, 1st in Salk Tournament (as of 2022/9/10).
• Track record is described in detail here

Jonathan Mann

• Professional in the financial sector
• GJOpen Brier score 0.155 vs. 0.247 median, ratio 0.63
• Pro forecaster on INFER
• About + Github

Juan Cambeiro

• MPH student in epidemiology at Columbia University
• Biosecurity fellow at the Institute for Progress
• Good Judgment Superforecaster
• Analyst for Metaculus
• GJOpen Brier score 0.25 vs. 0.317 median, ratio 0.79
• First in IARPA's FOCUS Tournament on COVID-19 forecasting
• First in Good Judgment Open's US Election 2020 challenge
• Second in Good Judgment Open's In the News 2020 challenge
• First in the time active on Good Judgment Open's Coronavirus Outbreak challenge, now third

Sam Glover

• Writes about social science at samstack.io
• Good Judgment Superforecaster
• GJOpen Brier score 0.228vs 0.309 median, ratio 0.74, increased significantly since becoming inactive in late 2021

Molly Hickman

• Data scientist at nLine, Inc., and previously worked on crowd-sourced prediction projects on the evaluation side.
• Top 2 forecaster of all time on INFER (as of 2022/9/15); 2nd place forecaster in season 2021 (-1.684 relative Brier score); 8th place forecaster in season 2020 (-0.529 relative Brier score)

Gregory Lewis

• Ex public health doctor, now working in biosecurity
• Good Judgment Superforecaster (since 2021)
• GJOpen Brier score 0.226 vs. 0.313 median, ratio 0.72

Pablo Stafforini

• Philosopher by training, currently editor of the Global Priorities Encyclopedia.
• 1st place in Road to Recovery tournament
• 3rd place in Li Wenliang Forecasting Tournament I tournament
• 1st place in Li Wenliang Forecasting Tournament II tournament
• See my Metaculus profile for additional information.

Jared Leibowich

• Data Scientist
• Forecaster for the Swift Centre
• GJO Brier score 0.22 vs. 0.303 median, ratio 0.73
• Placed 1st out of 7,000 "In the News 2021" Good Judgment competition
• Currently ranked 1st out of 3,000 "In the News 2022" Good Judgment competition

Greg Justice

• Analyst in the healthcare space
• GJOpen Brier score 0.187 vs. 0.251 median, ratio 0.75
• 1st place in Adam Grant's Think Again challenge on GJ Open (as of 2022-09-11)
• 4th place - Olivier Sibony's Noise challenge on GJ Open (as of 2022-09-11)
• 7th place - Good Judgment Project 2.0, in initial accuracy (as of 2022-09-11)

@belikewater

• MA in Economics, PhD in Neuroscience
• GJOpen Brier score 0.215 vs. 0.279 median, ratio 0.77
• Recently, also forecasting at INFER, Metaculus, Manifold Markets

Tolga Bilge

• Mathematics student
• GJI Superforecaster
• Forecaster for the Swift Centre
• GJOpen Brier score 0.164 vs. 0.254 median, ratio 0.65
• Forecaster on INFER