personal/nunosempere.com
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat: add article

Browse Source
This commit is contained in:
Nuno Sempere 2022-11-09 16:52:48 +00:00
parent c9043736d0
commit 7ccdaadc60
2 changed files with 388 additions and 117 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

232
blog/2022/11/06/forecasting-money-flows/.old/index.md Normal file
View File

@ -0,0 +1,232 @@
Tracking the money flows in forecasting
=======================================
This list of forecasting organizations includes:
- A brief description of each organization
- A monetary estimate of value. This can serve as a rough but hard-to-fake proxy of value. Sometimes this is a flow (e.g., budget per year), and sometimes this is an estimate of total value (e.g., valuation).
- A more subjective, rough, and verbal estimate of how much value the organization produces. <p><figure><img src="https://i.imgur.com/gqCTHMq.png" class="img-frontpage-center"><br><figcaption>DALLE: "crystal ball surrounded by money, photorealistic"</figcaption></figure></p>
This started as a breadth first evaluation of the forecasting system, and to some extent it still is, i.e., it might be useful to get a rough sense of the ecosystem as a whole. After some discussion on whether very rough evaluations [are worth it](https://nunosempere.com/blog/2022/10/27/are-flimsy-evaluations-worth-it/) ([a](http://web.archive.org/web/20221031125900/https://nunosempere.com/blog/2022/10/27/are-flimsy-evaluations-worth-it/)), people who prefer their evaluations to have a high threshold of quality and polish might want to either ignore this post or just pay attention at the monetary estimates.
## Larger entities
### Flutter Entertainment
***What it is***: [Flutter Entertainment](https://en.wikipedia.org/wiki/Flutter_Entertainment) ([a](http://web.archive.org/web/20221013031739/https://en.wikipedia.org/wiki/Flutter_Entertainment)) is one of the largest gambling companies. It was created from the merger of [Betfair](https://en.wikipedia.org/wiki/Betfair) ([a](http://web.archive.org/web/20220913120222/https://en.wikipedia.org/wiki/Betfair)) and [Paddy Power](https://en.wikipedia.org/wiki/Paddy_Power) ([a](http://web.archive.org/web/20220809000831/https://en.wikipedia.org/wiki/Paddy_Power)). It has a large presence in the EU and UK.
***Monetary value (market cap)***: It has a ~$20B (~£18B) market cap.
***Social value***: Betfair and Paddy Power have some markets with public utilities, e.g., around elections, but focus mostly on sports. However, its social value is otherwise low or negative---e.g., by enabling or creating gambling addicts.
***Note***: There are many other betting companies, e.g., Draft Kings, which I will not review here because they tend to focus mostly on sports, though I will note that they tend to have a high market capitalization.
### Kalshi
***What it is***: Kalshi is a US company which aims to provide [prediction markets](https://en.wikipedia.org/wiki/Prediction_market) ([a](http://web.archive.org/web/20221028173838/https://en.wikipedia.org/wiki/Prediction_market)) to US consumers while complying and shaping US regulations on this topic.
***Monetary value (VC funding, valuation)***: [Kalshi](https://kalshi.com/) ([a](http://web.archive.org/web/20221101033945/https://kalshi.com/)) has received ~$30M in VC [funding](https://www.crunchbase.com/organization/kalshi/company_financials). My sense is that it's probably worth much more than that ($100M+) due to being the first prediction market regulated by the CFTC. Previous markets---like PredictIt or Iowa Electronic Markets---were only operating with a no-action letter.
***Social value***: There has been a bit of drama around Kalshi recently. In recent times, the CFTC withdrew its letter of no action from PredictIt, and fined and banned Polymarket from operating in the US. These are two of Kalshi's competitors, and my sense is that Kalshi probably contributed to these events. As such, I'd estimate Kalshi's impact so far to probably be negative, on account of these anti-competitive practices. In the future, however, if Kalshi contributes to the adoption of prediction markets in the US, and this leads to better decision-making, it could end up having a large positive value. It is also possible that their contracts have value as hedging instruments.
### Metaculus
***What it is***: [Metaculus](https://www.metaculus.com/) is a forecasting site and a public benefit corporation that spearheads forecasting initiatives.
***Monetary value (grant flow)***: Recently, Open Philanthropy granted them [$5.5M](https://www.openphilanthropy.org/grants/?q=Metaculus) ([a](https://web.archive.org/web/20221106215746/https://www.openphilanthropy.org/grants/?q=Metaculus)). With a 5% discount rate, and asuming the $5.5M grant is for 2 years will be indefinitely renewed, this makes[^1] Metaculus' discounted cashflow be worth $55M. Metaculus previously received [$308k](https://funds.effectivealtruism.org/funds/payouts/may-august-2021-ea-infrastructure-fund-grants) ([a](http://web.archive.org/web/20220804053231/https://funds.effectivealtruism.org/funds/payouts/may-august-2021-ea-infrastructure-fund-grants)) from the EA infrastructure Fund in 2021 and [$65k](https://funds.effectivealtruism.org/funds/payouts/april-2020-long-term-future-fund-grants-and-recommendations) ([a](http://web.archive.org/web/20220806101148/https://funds.effectivealtruism.org/funds/payouts/april-2020-long-term-future-fund-grants-and-recommendations)) from the EA Long-term Future Fund in 2020.
[^1]: This form of discounted cash flow might be useful for some very rough comparison between for-profits and not-for-profits. But they really are not the same.
***Social value***: With its $5.5M Open Philanthropy grant, Metaculus has been growing its already [12-person-strong team](https://www.metaculus.com/about/) ([a](http://web.archive.org/web/20220925082358/https://www.metaculus.com/about/)), and is hiring for [a number of positions](https://apply.workable.com/metaculus/) ([a](http://web.archive.org/web/20221105123354/https://apply.workable.com/metaculus/)). So it's one of the EA forecasting organizations most able to scale and deploy human capital in terms of sheer number. The community which makes up its forecasting site is also reasonably strong.
### Polymarket
***What it is:*** Polymarket is a crypto prediction market. It combined various technologies: [magic.link](https://magic.link/), [Polygon](https://polygon.technology/) ([a](http://web.archive.org/web/20221106192328/https://polygon.technology/)) and the [Gnosis conditional contract](https://blog.gnosis.pm/omen-and-the-next-generation-of-prediction-markets-2e7a2dd604e) in order to build a usable crypto prediction market at a time where the previous iteration, Augur, had become unusable in practice due to high fees on the Ethereum blockchain.
***Monetary value (VC funding)***: It has received [$4M in VC funding](https://www.crunchbase.com/organization/polymarket) ([a](http://web.archive.org/web/20220428173056/https://www.crunchbase.com/organization/polymarket)), according to Crunchbase. It's unclear to me what its valuation is.
***Social value***: Polymarket has had a range of public interest markets, on politics, COVID infection numbers, or the Russian invasion of Ukraine.
### Cultivate Labs
***What it is***: Cultivate Labs is a company which offers infrastructure for forecasting tournaments. They currently host Good Judgment Open, INFER, the Cosmic Bazaar, and a Czech Prediction platform---and probably a few more platforms that I don't know of.
***Monetary value (valuation)***: I'm pretty uncertain about how much they are worth, and I'd give a guess of $5M to $80M.
***Social value***: I used to view them as a bit clunky and outdated, but I've come to respect that they are one of the only few groups willing to offer a full package, with support for multiple years.
### Good Judgment
***What it is***: Good Judgment" I mean the set of organizations which administer the public prediction market tournament [GJOpen](https://www.gjopen.com/) ([a](http://web.archive.org/web/20221101071604/https://www.gjopen.com/)), and provide forecasts and training through the [Good Judgment brand](https://goodjudgment.io/) ([a](http://web.archive.org/web/20221021191540/https://goodjudgment.io/))
***Monetary value (valuation)***: I'm fairly uncertain how much they would be worth. Maybe a very wide interval would be $3M to $50M.
***Social value***: Good Judgment has helped build awareness of superforecasting practices. Their forecasts provided to their subscribers have probably influenced the decisions of their readers and of those who commissioned them. Their training at government and corporations probably marginally increased the decision quality of clients, but it's unclear to me to what extent and for how long.
### Manifold Markets
***What it is:*** Manifold Market is a play-money prediction market site with a really nice user experience.
***Monetary value (VC funding, valuation):*** Manifold Market has received $2M in funding, ([$2M in seed funding](https://manifold.markets/ManifoldMarkets/will-manifold-raise-2m-in-seed-fund) ([a](http://web.archive.org/web/20221031122017/https://manifold.markets/ManifoldMarkets/will-manifold-raise-2m-in-seed-fund)), of which $[1M from the FTX Future Fund's regrantor program](https://ftxfuturefund.org/our-regrants/?_search=manifold%20markets) ([a](https://web.archive.org/web/20221106215835/https://ftxfuturefund.org/our-regrants/?_search=manifold%20markets))), at a $15M valuation. They have a superb software engineering team, and one of the fastest development speeds, but on the other hand their user acquisition has been a bit [slow](https://manifold.markets/stats) ([a](http://web.archive.org/web/20221031003326/https://manifold.markets/stats)). My sense is that their valuation would be somewhere between $10M and $50M
***Social value:*** So far, by making an eminently usable site, they allow users to signal their beliefs and challenge others to do the same. They have also hosted a few tournaments, and at least the [Clearer Thinking tournament](https://manifold.markets/tournaments) ([a](https://web.archive.org/web/20221106215934/https://manifold.markets/tournaments)) was probably decision-relevant. But their value is linked to their future growth.
### Insight Prediction
***What it is***: [Insight Prediction](https://insightprediction.com/) ([a](http://web.archive.org/web/20221101064147/https://insightprediction.com/)) is a new prediction market which accepts crypto.
***Monetary value (valuation)***: This would depend on their usage numbers, which aren't public. I'd say it's worth at least $5M, but whether the upper bound is $50M or $500M probably depends on the team's execution over the coming year.
***Social value***: It is one of the few prediction sites which hosts real-money somewhat liquid markets on the Ukraine-Russia conflict.
### Tetlock's research group
***What it is***: Tetlock's research group is a loose collaboration of academics working in collaboration with [Phil Tetlock](https://scholar.google.com/citations?user=CJjf6H0AAAAJhttps://scholar.google.com/citations?user=CJjf6H0AAAAJ&hl=en&oi=ao) ([a](https://web.archive.org/web/20221106220007/https://scholar.google.com/citations?user=CJjf6H0AAAAJhttps://scholar.google.com/citations?user=CJjf6H0AAAAJ&hl=en&oi=ao))---author of the ***Superforecasting*** book.
***Monetary value (valuation, grant flow)***: They have received at least $2M from Open Philanthropy ([1](https://www.openphilanthropy.org/grants/university-of-pennsylvania-philip-tetlocks-making-conversations-smarter-faster-forecasting-project/) ([a](http://web.archive.org/web/20221017005338/https://www.openphilanthropy.org/grants/university-of-pennsylvania-philip-tetlocks-making-conversations-smarter-faster-forecasting-project/)), [2](https://www.openphilanthropy.org/grants/university-of-pennsylvania-philip-tetlock-on-forecasting/) ([a](http://web.archive.org/web/20221017005338/https://www.openphilanthropy.org/grants/university-of-pennsylvania-philip-tetlock-on-forecasting/))), and probably more than that through the University of Pennsylvania. It's unclear to me how much to value a research group, but I'd give a ballpark of $1M to $20M.
***Social value***: Tetlock's work on expert political judgment and on superforecasting was seminal work that kick-started the current forecasting community. In [recent times](https://scholar.google.com/citations?hl=en&user=CJjf6H0AAAAJ&view_op=list_works&sortby=pubdate), it seems likely that work stemming from their [work on estimating existential risk](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4001628) ([a](http://web.archive.org/web/20220812234335/https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4001628)) could be action-guiding for large organizations and governmental bodies, but most of the output from that research direction hasn't been published yet.
### PredictIt
***What it is***: PredictIt is a US-based prediction market. Historically, a community of sharp prediction market players coalesced around it. But in recent times, the CFTC has withdrawn its permission for it to operate in the US.
***Monetary value (valuation)***: Insight Prediction gives it only an [11% chance of it surviving](https://insightprediction.com/m/18505/will-predictit-survive) ([a](http://web.archive.org/web/20221005064000/https://insightprediction.com/m/18505/will-predictit-survive)). My guess is that it was worth $5M to $50M when alive, implying a $0.5M to $5M valuation right now.
***Social value***: PredictIt was fairly instrumental in the creation of a passionate if sometimes a bit cut-throat prediction market community. Its future value depends on its survival.
### Epoch
***What it is***: [Epoch](https://epochai.org/) ([a](http://web.archive.org/web/20221013183722/https://epochai.org/)) is a research team investigating AI progress.
***Monetary value (grant flow):*** They have received around [$2M](https://www.openphilanthropy.org/grants/epoch-general-support/) ([a](http://web.archive.org/web/20220922190346/https://www.openphilanthropy.org/grants/epoch-general-support/)) from Open Philanthropy.
***Social value***: My sense is that their research has usefully informed some OpenPhilanthropy decisions. Open Philanthropy donated around $80M in 2021 to the "Potential Risks from Advanced AI" cause area. A 5% improvement in that decision-making would imply a value of $4M/year.
### Swift Centre For Applied Forecasting
***What it is***: The [Swift Centre](https://www.swiftcentre.org/) ([a](http://web.archive.org/web/20221103141937/https://www.swiftcentre.org/)) publishes publicly available forecasts from accurate forecasters on topics of importance.
***Monetary value (grant flow):*** They have received [$2M from the FTX future fund regranting program](https://ftxfuturefund.org/our-regrants/?_search=Swift) ([a](https://web.archive.org/web/20221106220034/https://ftxfuturefund.org/our-regrants/?_search=Swift)).
***Social value***: Their [forecasts](https://www.swiftcentre.org/) ([a](http://web.archive.org/web/20221103141937/https://www.swiftcentre.org/)) lead to better decisions. My sense is that the organization is still getting up to speed, and that when running at full operational capacity, they'll be producing many more predictions.
### Quantified Uncertainty Research Institute (QURI)
***What it is***: QURI is a "new initiative to advance forecasting and epistemics to improve the long-term future of humanity. We write research and make software." In particular, we are developing [Squiggle](https://squiggle-language.com/), a web-capable programming language for better estimation, and doing research around scalable forecasting and estimation.
***Monetary value (grants flow)***: Overall we have received around $780k so far ([1](https://survivalandflourishing.fund/sff-2020-h1-recommendations) ([a](http://web.archive.org/web/20220703183731/https://survivalandflourishing.fund/sff-2020-h1-recommendations)), [2](https://survivalandflourishing.fund/sff-2022-h1-recommendations) ([a](http://web.archive.org/web/20220703184038/https://survivalandflourishing.fund/sff-2022-h1-recommendations)), [3](https://survivalandflourishing.fund/sff-2022-h1-recommendations) ([a](http://web.archive.org/web/20220703184038/https://survivalandflourishing.fund/sff-2022-h1-recommendations))), maybe a bit more.
***Social value***: Most of QURI's value is coming from the value of [Squiggle](https://www.squiggle-language.com/) ([a](http://web.archive.org/web/20221012160532/https://www.squiggle-language.com/)), which people are now starting to use for cost-effectiveness estimates, and from the value of [my own research](https://forum.effectivealtruism.org/users/nunosempere) ([a](http://web.archive.org/web/20221106162329/https://forum.effectivealtruism.org/users/nunosempere)), which might move the Effective Altruism movement more in the direction of using quantified estimates even for speculative stuff.
***Note***: This is the organization that I work for.
### Samotsvety Forecasting
***What it is***: [Samotsvety Forecasting](https://samotsvety.org/) ([a](http://web.archive.org/web/20221025004219/https://samotsvety.org/)) is a somewhat ad-hoc group of highly accurate forecasters of which I am a member.
***Monetary value***: My sense is that we could find some funder to sacrifice $200k to $5M if our group's existence depended on it. But I haven't actually tried that!
***Social value:*** We've published a variety of public forecasts, and we find ourselves more and more to be a part of Effective Altruism's general epistemic infrastructure.
### Czech Priorities
***What it is***: [Czech Priorities](https://www.ceskepriority.cz/) ([a](http://web.archive.org/web/20221102103806/https://www.ceskepriority.cz/)) is a Czech group working to advance forecasting within the Czech Republic.
***Monetary value:*** I couldn't find grantmaking information from a quick online search.
***Social value***: It's possible that they could find generalizable lessons about using forecasting to influence policy and improve decision-making that could then be extended to other countries. But this is not certain.
### Hedgehog Markets
***What it is***: [Hedghog Markets](https://hedgehog.markets/) ([a](http://web.archive.org/web/20221010202946/https://hedgehog.markets/)) is a crypto-prediction market
***Monetary value***: They raised [$3.5M in funding](https://www.crunchbase.com/organization/hedgehog-markets) back in the crypto bull days.
***Social value***: They have been particularly innovative around no-loss markets, and peer-to-peer markets. But value depends on future usage volume.
### INFER
***What it is***: INFER is a prediction market platform which seeks to influence US government policy around transformative technologies.
***Monetary value***: INFER is hosted at [ARLIS](https://www.arlis.umd.edu/home) ([a](http://web.archive.org/web/20221013111200/https://www.arlis.umd.edu/home)), at the University of Maryland, which received an ~[$8M grant from Open Philanthropy specifically for forecasting](https://www.openphilanthropy.org/grants/applied-research-laboratory-for-intelligence-and-security-forecasting-platforms/) ([a](http://web.archive.org/web/20220808081108/https://www.openphilanthropy.org/grants/applied-research-laboratory-for-intelligence-and-security-forecasting-platforms/)) (over two years, over two platforms), which I find very confusing because I don't think they have deployed much of that capital yet.
***Social value***: INFER aims to influence decisions by the US government around technology policy. To the extent that this is the case, it could be valuable indeed, but this is hard to evaluate from the outside.
## Smaller forecasting entities
### Nathan Young
[Nathan Young](https://twitter.com/NathanpmYoung) is a smallish forecasting influencer followed by 6k people on Twitter. He recently got [$182k from the FTX Future Fund regranting program](https://ftxfuturefund.org/our-grants/?_search=Nathan%20Young) ([a](https://web.archive.org/web/20221106220135/https://ftxfuturefund.org/our-grants/?_search=Nathan%20Young)) to build a platform for forecasting question generation, an alpha version of which can be seen [here](https://doubtful.app/) ([a](http://web.archive.org/web/20221031061542/https://doubtful.app/)).
### Sage
Sage is an organization dedicated to forecasting tooling and research. It got $700k from the FTX Future Fund, and has built the tools on [quantifiedintuitions.org](https://www.quantifiedintuitions.org/pastcasting) ([a](http://web.archive.org/web/20221013134558/https://www.quantifiedintuitions.org/pastcasting)). But it hasn't deployed most of its capital yet.
### Global Guessing
[Global Guessing](https://globalguessing.com/) ([a](http://web.archive.org/web/20221031010912/https://globalguessing.com/)) is a forecasting site which made and gathered forecasts early on the Russian invasion of Ukraine, but which seems pretty dormant since then. They have received [~$330k from the FTX Future Fund.](https://ftxfuturefund.org/our-regrants/?_search=Global%20Guessing) ([a](https://web.archive.org/web/20221106220157/https://ftxfuturefund.org/our-regrants/?_search=Global%20Guessing))
### Social Science Prediction Platform
The [Social Science Prediction Platform](https://socialscienceprediction.org/) ([a](http://web.archive.org/web/20221013155854/https://socialscienceprediction.org/)) collects forecasts from academics on upcoming papers. This can have a range of public benefits, such as estimates of how surprising results in the social sciences are. Personally I'd be hopeful about the ability to construct a more legible or objective Bayesian prior. I could quickly find that they have received [$346k](https://survivalandflourishing.fund/sff-2022-h1-recommendations) ([a](http://web.archive.org/web/20220703184038/https://survivalandflourishing.fund/sff-2022-h1-recommendations)) from the Survival and Flourishing Fund.
### Augur
[Augur](https://augur.net/) was a pioneering prediction market way back when, but isn't up to much these days. Somehow the implied [market cap](https://coinmarketcap.com/currencies/augur/) ([a](http://web.archive.org/web/20220619100141/https://coinmarketcap.com/currencies/augur/)) of their coin is still $74M (## ?!?).
### Confido
[Confido](https://confido.tools/) ([a](http://web.archive.org/web/20220711161543/https://confido.tools/)) is a three-person strong Czech organization which develops forecasting tooling. They received $190k from the [FTX Future Fund regranting program](https://ftxfuturefund.org/our-regrants/?_search=Confido) ([a](https://web.archive.org/web/20221106220218/https://ftxfuturefund.org/our-regrants/?_search=Confido)).
### Hypermind
[Hypermind ](https://www.hypermind.com/en/) ([a](http://web.archive.org/web/20221015015526/https://www.hypermind.com/en/))is a French forecasting organization which hosts some low-stakes markets with play money but real-money rewards.
### Replication Markets
[Replication Markets](https://replicationmarkets.com/) ([a](http://web.archive.org/web/20220930085636/https://replicationmarkets.com/)) was a super-interesting experiment in having forecasters predict the outcome of replications. It has now ended, and it's uncertain whether the creators will follow up with something as interesting. They had ~$150k in forecaster rewards.
### PredictionBook
[PredictionBook ](https://predictionbook.com/) ([a](http://web.archive.org/web/20221027154039/https://predictionbook.com/))is an old, old site used by people to keep track of their probabilities, with no rewards involved. Gwern still [uses it](https://predictionbook.com/users/gwern) ([a](http://web.archive.org/web/20221019042049/https://predictionbook.com/users/gwern)).
### Gnosis
Gnosis is much like Augur, but less successful. They also have a [GnosisDAO](https://gnosis.io/gnosisdao/) ([a](http://web.archive.org/web/20220825162029/https://gnosis.io/gnosisdao/)). The implied market cap of their token is worth [$330M](https://gnosis.io/gnosisdao/) ([a](http://web.archive.org/web/20220825162029/https://gnosis.io/gnosisdao/)), based on the [Gnosis safe](https://gnosis-safe.io/) ([a](http://web.archive.org/web/20221101042815/https://gnosis-safe.io/)). But it doesn't do much forecasting these days.
## Thoughts on this ranking
### On the money flows
Money flows are interesting, but flawed as a measure of value. The spending rate or VC money raised does not correspond to the value a project creates. The high valuations for crypto projects could be mostly illusory, and for the case of Gnosis and Augur, they probably are.
Still, a monetary ranking has some advantages
- There may be some ambiguity to it, but it allows one to differentiate between &lt;$500k projects, $500k to $5M projects, and $5M to $50M projects.
- It has an honesty to it, in that money is a somewhat hard-to-fake signal
- It is a fast proxy to estimate
### On the estimate of social value
The estimate of social value isn't a quantified estimate, but rather a few lines of observations, and maybe a path to impact. I think I could translate this into a numerical relative value scale, but it would take a fair amount of time.
### On the different pathways to impact
I'm seeing mainly three pathways of impact for forecasting projects:
1. Providing consumer value, e.g., betting markets or hedging utilities that people want to use
2. Providing informative markets about events of interest, which can apply pressure to decision-makers inside governments and corporations to make better decisions
3. Providing tooling that people making decisions can make and incorporate as part of their decision-making
I'm mostly bullish on the third option, but this deserves more elaboration.
### On how this estimate could be improved
This estimate could be improved by having numerical estiamates of impact. This would allow for comparisons around efficiency &c. It could also be presented better, i.e., in a table, or more creatively, in a map.
<p><section id="isso-thread">
<noscript>Javascript needs to be activated to view comments.</noscript>
</section></p>
</article>

273
blog/2022/11/06/forecasting-money-flows/index.md
View File

@ -1,231 +1,270 @@
Tracking the money flows in forecasting
=======================================
Tracking the money flows in forecasting
==============
This list of forecasting organizations includes:
This list of forecasting organizations includes:
- A brief description of each organization
- A monetary estimate of value. This can serve as a rough but hard-to-fake proxy of value. Sometimes this is a flow (e.g., budget per year), and sometimes this is an estimate of total value (e.g., valuation).
- A more subjective, rough, and verbal estimate of how much value the organization produces. <p><figure><img src="https://i.imgur.com/gqCTHMq.png" class="img-frontpage-center"><br><figcaption>DALLE: "crystall ball surrounded by money, photorealistic"</figcaption></figure></p>
* A brief description of each organization
* A monetary estimate of value. This can serve as a rough but hard-to-fake proxy of value. Sometimes this is a flow (e.g., budget per year), and sometimes this is an estimate of total value (e.g., valuation).
* A more subjective, rough, and verbal estimate of how much value the organization produces. <p><figure><img src="https://i.imgur.com/gqCTHMq.png" class="img-frontpage-center"><br><figcaption>DALLE: "crystal ball surrounded by money, photorealistic"</figcaption></figure></p>
This started as a breadth first evaluation of the forecasting system, and to some extent it still is, i.e., it might be useful to get a rough sense of the ecosystem as a whole. After some discussion on whether very rough evaluations [are worth it](https://nunosempere.com/blog/2022/10/27/are-flimsy-evaluations-worth-it/) ([a](http://web.archive.org/web/20221031125900/https://nunosempere.com/blog/2022/10/27/are-flimsy-evaluations-worth-it/)), people who prefer their evaluations to have a high threshold of quality and polish might want to either ignore this post or just pay attention at the monetary estimates.
This started as a breadth first evaluation of the forecasting system, and to some extent it still is, i.e., it might be useful to get a rough sense of the ecosystem as a whole. After some discussion on whether very rough evaluations [are worth it](https://nunosempere.com/blog/2022/10/27/are-flimsy-evaluations-worth-it/) ([a](http://web.archive.org/web/20221031125900/https://nunosempere.com/blog/2022/10/27/are-flimsy-evaluations-worth-it/)), people who prefer their evaluations to have a high threshold of quality and polish might want to either ignore this post or just pay attention to the monetary estimates.
## Larger entities
## Summary table
### Flutter Entertainment
Note that the monetary value column has different types of estimates. 
***What it is***: [Flutter Entertainment](https://en.wikipedia.org/wiki/Flutter_Entertainment) ([a](http://web.archive.org/web/20221013031739/https://en.wikipedia.org/wiki/Flutter_Entertainment)) is one of the largest gambling companies. It was created from the merger of [Betfair](https://en.wikipedia.org/wiki/Betfair) ([a](http://web.archive.org/web/20220913120222/https://en.wikipedia.org/wiki/Betfair)) and [Paddy Power](https://en.wikipedia.org/wiki/Paddy_Power) ([a](http://web.archive.org/web/20220809000831/https://en.wikipedia.org/wiki/Paddy_Power)). It has a large presence in the EU and UK.
| Name | Monetary value | Purpose |
|-------------------------------------------|-----------------------------------------|----------------------------------------------------|
| Flutter Entertainment | ~$20B (~£18B) market cap | Gambling |
| Kalshi | ~$30M in VC funding | Prediction market |
| Metaculus | ~$6M in grants | Forecasting site |
| Polymarket | ~$4M in VC funding | Prediction market |
| Cultivate Labs | $5M to $80M (estimated valuation) | Forecasting platform as a service |
| Good Judgment | $3M to $50M (estimated valuation) | Forecasting consulting |
| Manifold Markets | ~$2M in early stage funding | Play money prediction market |
| Insight Prediction | — | Prediction market |
| Tetlock’s research group | $1M to $20M (estimated grant flow) | Research group |
| PredictIt | $0.5M to $5M (estimated value) | Prediction market |
| Epoch | $2M in grants | Research group on AI progress |
| Swift Centre | $2M in grants | Forecasts as a public good |
| Quantified Uncertainty Research Institute | $780k in grants | Software and forecasting research |
| Samotsvety Forecasting | $200k to $5M (estimated valuation) | Forecasts as a public good, forecasting consulting |
| Czech Priorities | — | Institutional decision-making using forecasting |
| Hedgehog Markets | $3.5M in VC funding | Prediction market (crypto) |
| INFER | Uncertain. Estimated $2M/year in grants | Forecasting platform |
| Nathan Young | $180k in grants | Forecasting question creation |
| Sage | $700k in grants | Forecasting research |
| Global Guessing | $330k in grants | Forecasting journalism |
| Social Science Prediction Platform | At least $838k in grants | Forecasting in an academic context |
| Augur | $60M crypto market cap | Prediction market |
| Confido | $190k in grants | Forecasting tooling |
| Hypermind | — | Play money prediction market |
| Replication Markets | $150k in forecaster rewards | Forecasting experiment |
| PredictionBook | — | Prediction database |
| Gnosis | $230M crypto market cap | Smart contracts, including for prediction markets |
***Monetary value (market cap)***: It has a ~$20B (~£18B) market cap.
***Social value***: Betfair and Paddy Power have some markets with public utilities, e.g., around elections, but focus mostly on sports. However, its social value is otherwise low or negative---e.g., by enabling or creating gambling addicts.
## Larger entities
***Note***: There are many other betting companies, e.g., Draft Kings, which I will not review here because they tend to focus mostly on sports, though I will note that they tend to have a high market capitalization.
### Flutter Entertainment
### Kalshi
_**What it is**_: [Flutter Entertainment](https://en.wikipedia.org/wiki/Flutter_Entertainment) ([a](http://web.archive.org/web/20221013031739/https://en.wikipedia.org/wiki/Flutter_Entertainment)) is one of the largest gambling companies. It was created from the merger of [Betfair](https://en.wikipedia.org/wiki/Betfair) ([a](http://web.archive.org/web/20220913120222/https://en.wikipedia.org/wiki/Betfair)) and [Paddy Power](https://en.wikipedia.org/wiki/Paddy_Power) ([a](http://web.archive.org/web/20220809000831/https://en.wikipedia.org/wiki/Paddy_Power)). It has a large presence in the EU and UK.
***What it is***: Kalshi is a US company which aims to provide [prediction markets](https://en.wikipedia.org/wiki/Prediction_market) ([a](http://web.archive.org/web/20221028173838/https://en.wikipedia.org/wiki/Prediction_market)) to US consumers while complying and shaping US regulations on this topic.
_**Monetary value (market cap)**_: It has a ~$20B (~£18B) market cap.
***Monetary value (VC funding, valuation)***: [Kalshi](https://kalshi.com/) ([a](http://web.archive.org/web/20221101033945/https://kalshi.com/)) has received ~$30M in VC [funding](https://www.crunchbase.com/organization/kalshi/company_financials). My sense is that it's probably worth much more than that ($100M+) due to being the first prediction market regulated by the CFTC. Previous markets---like PredictIt or Iowa Electronic Markets---were only operating with a no-action letter.
_**Social value**_: Betfair and Paddy Power have some markets with public utilities, e.g., around elections, but focus mostly on sports. However, its social value is otherwise low or negative—e.g., by enabling or creating gambling addicts.
***Social value***: In recent times, the CFTC withdrew its letter of no action from PredictIt, and fined and banned Polymarket from operating in the US. These are two of Kalshi's competitors, and my sense is that Kalshi probably contributed to these events. As such, I'd estimate Kalshi's impact so far to probably be negative, on account of these anti-competitive practices. In the future, however, if Kalshi contributes to the adoption of prediction markets in the US, and this leads to better decision-making, it could end up having a large positive value. It is also possible that their contracts have value as hedging instruments.
_**Note**_: There are many other betting companies, e.g., Draft Kings, which I will not review here because they tend to focus mostly on sports, though I will note that they tend to have a high market capitalization.
### Metaculus
### Kalshi
***What it is***: [Metaculus](https://www.metaculus.com/) is a forecasting site and a public benefit corporation that spearheads forecasting initiatives.
_**What it is**_: Kalshi is a US company which aims to provide [prediction markets](https://en.wikipedia.org/wiki/Prediction_market) ([a](http://web.archive.org/web/20221028173838/https://en.wikipedia.org/wiki/Prediction_market)) to US consumers while complying and shaping US regulations on this topic.
***Monetary value (grant flow)***: Recently, Open Philanthropy granted them [$5.5M](https://www.openphilanthropy.org/grants/?q=Metaculus) ([a](https://web.archive.org/web/20221106215746/https://www.openphilanthropy.org/grants/?q=Metaculus)). Assuming the $5.5M grant is for 2 years and a 5% discount rate, this makes Metaculus worth $55M. Metaculus previously received [$308k](https://funds.effectivealtruism.org/funds/payouts/may-august-2021-ea-infrastructure-fund-grants) ([a](http://web.archive.org/web/20220804053231/https://funds.effectivealtruism.org/funds/payouts/may-august-2021-ea-infrastructure-fund-grants)) from the EA infrastructure Fund in 2021 and [$65k](https://funds.effectivealtruism.org/funds/payouts/april-2020-long-term-future-fund-grants-and-recommendations) ([a](http://web.archive.org/web/20220806101148/https://funds.effectivealtruism.org/funds/payouts/april-2020-long-term-future-fund-grants-and-recommendations)) from the EA Long-term Future Fund in 2020.
_**Monetary value (VC funding, valuation)**_: [Kalshi](https://kalshi.com/) ([a](http://web.archive.org/web/20221101033945/https://kalshi.com/)) has received ~$30M in VC [funding](https://www.crunchbase.com/organization/kalshi/company_financials). My sense is that it’s probably worth much more than that ($100M+) due to being the first prediction market regulated by the CFTC. Previous markets—like PredictIt or Iowa Electronic Markets—were only operating with a no-action letter.
***Social value***: With its $5.5M Open Philanthropy grant, Metaculus has been growing its already [12-person-strong team](https://www.metaculus.com/about/) ([a](http://web.archive.org/web/20220925082358/https://www.metaculus.com/about/)), and is hiring for [a number of positions](https://apply.workable.com/metaculus/) ([a](http://web.archive.org/web/20221105123354/https://apply.workable.com/metaculus/)). So it's one of the EA forecasting organizations most able to scale and deploy human capital in terms of sheer number. The community which makes up its forecasting site is also reasonably strong.
_**Social value**_: There has been a bit of drama around Kalshi recently. In recent times, the CFTC withdrew its letter of no action from PredictIt, and fined and banned Polymarket from operating in the US. These are two of Kalshi’s competitors, and my sense is that Kalshi probably contributed to these events. As such, I’d estimate Kalshi’s impact so far to probably be negative, on account of these anti-competitive practices. In the future, however, if Kalshi contributes to the adoption of prediction markets in the US, and this leads to better decision-making, it could end up having a large positive value. It is also possible that their contracts have value as hedging instruments.
### Polymarket
### Metaculus
***What it is:*** Polymarket is a crypto prediction market. It combined various technologies: [magic.link](https://magic.link/), [Polygon](https://polygon.technology/) ([a](http://web.archive.org/web/20221106192328/https://polygon.technology/)) and the [Gnosis conditional contract](https://blog.gnosis.pm/omen-and-the-next-generation-of-prediction-markets-2e7a2dd604e) in order to build a usable crypto prediction market at a time where the previous iteration, Augur, had become unusable in practice due to high fees on the Ethereum blockchain.
_**What it is**_: [Metaculus](https://www.metaculus.com/) is a forecasting site and a public benefit corporation that spearheads forecasting initiatives.
***Monetary value (VC funding)***: It has received [$4M in VC funding](https://www.crunchbase.com/organization/polymarket) ([a](http://web.archive.org/web/20220428173056/https://www.crunchbase.com/organization/polymarket)), according to Crunchbase. It's unclear to me what its valuation is.
_**Monetary value (grant flow)**_: Recently, Open Philanthropy granted them [$5.5M](https://www.openphilanthropy.org/grants/?q=Metaculus) ([a](https://web.archive.org/web/20221106215746/https://www.openphilanthropy.org/grants/?q=Metaculus)). With a 5% discount rate, and asuming the $5.5M grant is for 2 years will be indefinitely renewed, this makes[^1] Metaculus' discounted cashflow be worth $55M. Metaculus previously received [$308k](https://funds.effectivealtruism.org/funds/payouts/may-august-2021-ea-infrastructure-fund-grants) ([a](http://web.archive.org/web/20220804053231/https://funds.effectivealtruism.org/funds/payouts/may-august-2021-ea-infrastructure-fund-grants)) from the EA infrastructure Fund in 2021 and [$65k](https://funds.effectivealtruism.org/funds/payouts/april-2020-long-term-future-fund-grants-and-recommendations) ([a](http://web.archive.org/web/20220806101148/https://funds.effectivealtruism.org/funds/payouts/april-2020-long-term-future-fund-grants-and-recommendations)) from the EA Long-term Future Fund in 2020.
***Social value***: Polymarket has had a range of public interest markets, on politics, COVID infection numbers, or the Russian invasion of Ukraine.
[^1]: This form of discounted cash flow might be useful for some very rough comparison between for-profits and not-for-profits. But they really are not the same.
### Cultivate Labs
_**Social value**_: With its $5.5M Open Philanthropy grant, Metaculus has been growing its already [12-person-strong team](https://www.metaculus.com/about/) ([a](http://web.archive.org/web/20220925082358/https://www.metaculus.com/about/)), and is hiring for [a number of positions](https://apply.workable.com/metaculus/) ([a](http://web.archive.org/web/20221105123354/https://apply.workable.com/metaculus/)). So it’s one of the EA forecasting organizations most able to scale and deploy human capital in terms of sheer number. The community which makes up its forecasting site is also reasonably strong.
***What it is***: Cultivate Labs is a company which offers infrastructure for forecasting tournaments. They currently host Good Judgment Open, INFER, the Cosmic Bazaar, and a Czech Prediction platform---and probably more a few more platforms that I don't know of.
### Polymarket
***Monetary value (valuation)***: I'm pretty uncertain about how much they are worth, and I'd give a guess of $5M to $80M.
_**What it is:**_ Polymarket is a crypto prediction market. It combined various technologies: [magic.link](https://magic.link/), [Polygon](https://polygon.technology/) ([a](http://web.archive.org/web/20221106192328/https://polygon.technology/)) and the [Gnosis conditional contract](https://blog.gnosis.pm/omen-and-the-next-generation-of-prediction-markets-2e7a2dd604e) in order to build a usable crypto prediction market at a time where the previous iteration, Augur, had become unusable in practice due to high fees on the Ethereum blockchain.
***Social value***: I used to view them as a bit clunky and outdated, but I've come to respect that they are one of the only few groups willing to offer a full package, with support for multiple years.
_**Monetary value (VC funding)**_: It has received [$4M in VC funding](https://www.crunchbase.com/organization/polymarket) ([a](http://web.archive.org/web/20220428173056/https://www.crunchbase.com/organization/polymarket)), according to Crunchbase. It’s unclear to me what its valuation is.
Good Judgment
_**Social value**_: Polymarket has had a range of public interest markets, on politics, COVID infection numbers, or the Russian invasion of Ukraine.
***What it is***: Good Judgment" I mean the set of organizations which administer the public prediction market tournament [GJOpen](https://www.gjopen.com/) ([a](http://web.archive.org/web/20221101071604/https://www.gjopen.com/)), and provide forecasts and training through the [Good Judgment brand](https://goodjudgment.io/) ([a](http://web.archive.org/web/20221021191540/https://goodjudgment.io/))
### Cultivate Labs
***Monetary value (valuation)***: I'm fairly uncertain how much they would be worth. Maybe a very wide interval would be $3M to $50M.
_**What it is**_: Cultivate Labs is a company which offers infrastructure for forecasting tournaments. They currently host Good Judgment Open, INFER, the Cosmic Bazaar, and a Czech Prediction platform—and probably a few more platforms that I don’t know of.
***Social value***: Good Judgment has helped build awareness of superforecasting practices. Their forecasts provided to their subscribers have probably influenced the decisions of their readers and of those who commissioned them. Their training at government and corporations probably marginally increased the decision quality of clients, but it's unclear to me to what extent and for how long.
_**Monetary value (valuation)**_: I’m pretty uncertain about how much they are worth, and I’d give a guess of $5M to $80M.
### Manifold Markets
_**Social value**_: I used to view them as a bit clunky and outdated, but I’ve come to respect that they are one of the only few groups willing to offer a full package, with support for multiple years.
***What it is:*** Manifold Market is a play-money prediction market site with a really nice user experience.
### Good Judgment
***Monetary value (VC funding, valuation):*** Manifold Market has received $2M in funding, ([$2M in seed funding](https://manifold.markets/ManifoldMarkets/will-manifold-raise-2m-in-seed-fund) ([a](http://web.archive.org/web/20221031122017/https://manifold.markets/ManifoldMarkets/will-manifold-raise-2m-in-seed-fund)), of which $[1M from the FTX Future Fund's regrantor program](https://ftxfuturefund.org/our-regrants/?_search=manifold%20markets) ([a](https://web.archive.org/web/20221106215835/https://ftxfuturefund.org/our-regrants/?_search=manifold%20markets))), at a $15M valuation. They have a superb software engineering team, and one of the fastest development speeds, but on the other hand their user acquisition has been a bit [slow](https://manifold.markets/stats) ([a](http://web.archive.org/web/20221031003326/https://manifold.markets/stats)). My sense is that their valuation would be somewhere between $10M and $50M
_**What it is**_: Good Judgment" I mean the set of organizations which administer the public prediction market tournament [GJOpen](https://www.gjopen.com/) ([a](http://web.archive.org/web/20221101071604/https://www.gjopen.com/)), and provide forecasts and training through the [Good Judgment brand](https://goodjudgment.io/) ([a](http://web.archive.org/web/20221021191540/https://goodjudgment.io/))
***Social value:*** So far, by making an eminently usable site, they allow users to signal their beliefs and challenge others to do the same. They have also hosted a few tournaments, and at least the [Clearer Thinking tournament](https://manifold.markets/tournaments) ([a](https://web.archive.org/web/20221106215934/https://manifold.markets/tournaments)) was probably decision-relevant. But their value is linked to their future growth.
_**Monetary value (valuation)**_: I’m fairly uncertain how much they would be worth. Maybe a very wide interval would be $3M to $50M.
### Insight Prediction
_**Social value**_: Good Judgment has helped build awareness of superforecasting practices. Their forecasts provided to their subscribers have probably influenced the decisions of their readers and of those who commissioned them. Their training at government and corporations probably marginally increased the decision quality of clients, but it’s unclear to me to what extent and for how long.
***What it is***: [Insight Prediction](https://insightprediction.com/) ([a](http://web.archive.org/web/20221101064147/https://insightprediction.com/)) is a new prediction market which accepts crypto.
### Manifold Markets
***Monetary value (valuation)***: This would depend on their usage numbers, which aren't public. I'd say it's worth at least $5M, but whether the upper bound is $50M or $500M probably depends on the team's execution over the coming year.
_**What it is:**_ Manifold Market is a play-money prediction market site with a really nice user experience.
***Social value***: It is one of the few prediction sites which hosts real-money somewhat liquid markets on the Ukraine-Russia conflict.
_**Monetary value (VC funding, valuation):**_ Manifold Market has received $2M in funding, ([$2M in seed funding](https://manifold.markets/ManifoldMarkets/will-manifold-raise-2m-in-seed-fund) ([a](http://web.archive.org/web/20221031122017/https://manifold.markets/ManifoldMarkets/will-manifold-raise-2m-in-seed-fund)), of which $[1M from the FTX Future Fund’s regrantor program](https://ftxfuturefund.org/our-regrants/?_search=manifold%20markets) ([a](https://web.archive.org/web/20221106215835/https://ftxfuturefund.org/our-regrants/?_search=manifold%20markets))), at a $15M valuation. They have a superb software engineering team, and one of the fastest development speeds, but on the other hand their user acquisition has been a bit [slow](https://manifold.markets/stats) ([a](http://web.archive.org/web/20221031003326/https://manifold.markets/stats)). My sense is that their valuation would be somewhere between $10M and $50M
### Tetlock's research group
_**Social value:**_ So far, by making an eminently usable site, they allow users to signal their beliefs and challenge others to do the same. They have also hosted a few tournaments, and at least the [Clearer Thinking tournament](https://manifold.markets/tournaments) ([a](https://web.archive.org/web/20221106215934/https://manifold.markets/tournaments)) was probably decision-relevant. But their value is linked to their future growth.
***What it is***: Tetlock's research group is a loose collaboration of academics working in collaboration with [Phil Tetlock](https://scholar.google.com/citations?user=CJjf6H0AAAAJhttps://scholar.google.com/citations?user=CJjf6H0AAAAJ&hl=en&oi=ao)hl=enhttps://scholar.google.com/citations?user=CJjf6H0AAAAJ&hl=en&oi=ao)oi=ao) ([a](https://web.archive.org/web/20221106220007/https://scholar.google.com/citations?user=CJjf6H0AAAAJhttps://scholar.google.com/citations?user=CJjf6H0AAAAJ&hl=en&oi=ao)hl=enhttps://scholar.google.com/citations?user=CJjf6H0AAAAJ&hl=en&oi=ao)oi=ao))---author of the ***Superforecasting*** book.
### Insight Prediction
***Monetary value (valuation, grant flow)***: They have received at least $2M from Open Philanthropy ([1](https://www.openphilanthropy.org/grants/university-of-pennsylvania-philip-tetlocks-making-conversations-smarter-faster-forecasting-project/) ([a](http://web.archive.org/web/20221017005338/https://www.openphilanthropy.org/grants/university-of-pennsylvania-philip-tetlocks-making-conversations-smarter-faster-forecasting-project/)), [2](https://www.openphilanthropy.org/grants/university-of-pennsylvania-philip-tetlock-on-forecasting/) ([a](http://web.archive.org/web/20221017005338/https://www.openphilanthropy.org/grants/university-of-pennsylvania-philip-tetlock-on-forecasting/))), and probably more than that through the University of Pennsylvania. It's unclear to me how much to value a research group, but I'd give a ballpark of $1M to $20M.
_**What it is**_: [Insight Prediction](https://insightprediction.com/) ([a](http://web.archive.org/web/20221101064147/https://insightprediction.com/)) is a new prediction market which accepts crypto.
***Social value***: Tetlock's work on expert political judgment and on superforecasting was seminal work that kick-started the current forecasting community. In [recent times](https://scholar.google.com/citations?hl=en&user=CJjf6H0AAAAJ&view_op=list_works&sortby=pubdate), it seems likely that work stemming from their [work on estimating existential risk](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4001628) ([a](http://web.archive.org/web/20220812234335/https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4001628)) could be action-guiding for large organizations and governmental bodies, but most of the output from that research direction hasn't been published yet.
_**Monetary value (valuation)**_: This would depend on their usage numbers, which aren’t public. I’d say it’s worth at least $5M, but whether the upper bound is $50M or $500M probably depends on the team’s execution over the coming year.
### PredictIt
_**Social value**_: It is one of the few prediction sites which hosts real-money somewhat liquid markets on the Ukraine-Russia conflict.
***What it is***: PredictIt is a US-based prediction market. Historically, a community of sharp prediction market players coalesced around it. But in recent times, the CFTC has withdrawn its permission for it to operate in the US.
### Tetlock’s research group
***Monetary value (valuation)***: Insight Prediction gives it only a [23% chance of it surviving](https://insightprediction.com/m/18505/will-predictit-survive) ([a](http://web.archive.org/web/20221005064000/https://insightprediction.com/m/18505/will-predictit-survive)). My guess is that it was worth $5M to $50M when alive, implying a $1M to $12M valuation right now.
_**What it is**_: Tetlock’s research group is a loose collaboration of academics working in collaboration with [Phil Tetlock](https://scholar.google.com/citations?user=CJjf6H0AAAAJhttps://scholar.google.com/citations?user=CJjf6H0AAAAJ&hl=en&oi=ao) ([a](https://web.archive.org/web/20221106220007/https://scholar.google.com/citations?user=CJjf6H0AAAAJhttps://scholar.google.com/citations?user=CJjf6H0AAAAJ&hl=en&oi=ao))—author of the _**Superforecasting**_ book.
***Social value***: PredictIt was fairly instrumental in the creation of a passionate if sometimes a bit cut-throat prediction market community. Its future value depends on its survival.
_**Monetary value (valuation, grant flow)**_: They have received at least $2M from Open Philanthropy ([1](https://www.openphilanthropy.org/grants/university-of-pennsylvania-philip-tetlocks-making-conversations-smarter-faster-forecasting-project/) ([a](http://web.archive.org/web/20221017005338/https://www.openphilanthropy.org/grants/university-of-pennsylvania-philip-tetlocks-making-conversations-smarter-faster-forecasting-project/)), [2](https://www.openphilanthropy.org/grants/university-of-pennsylvania-philip-tetlock-on-forecasting/) ([a](http://web.archive.org/web/20221017005338/https://www.openphilanthropy.org/grants/university-of-pennsylvania-philip-tetlock-on-forecasting/))), and probably more than that through the University of Pennsylvania. It’s unclear to me how much to value a research group, but I’d give a ballpark of $1M to $20M.
### Epoch
_**Social value**_: Tetlock’s work on expert political judgment and on superforecasting was seminal work that kick-started the current forecasting community. In [recent times](https://scholar.google.com/citations?hl=en&user=CJjf6H0AAAAJ&view_op=list_works&sortby=pubdate), it seems likely that work stemming from their [work on estimating existential risk](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4001628) ([a](http://web.archive.org/web/20220812234335/https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4001628)) could be action-guiding for large organizations and governmental bodies, but most of the output from that research direction hasn’t been published yet.
***What it is***: [Epoch](https://epochai.org/) ([a](http://web.archive.org/web/20221013183722/https://epochai.org/)) is a research team investigating AI progress.
### PredictIt
***Monetary value (grant flow):*** They have received around [$2M](https://www.openphilanthropy.org/grants/epoch-general-support/) ([a](http://web.archive.org/web/20220922190346/https://www.openphilanthropy.org/grants/epoch-general-support/)) from Open Philanthropy.
_**What it is**_: PredictIt is a US-based prediction market. Historically, a community of sharp prediction market players coalesced around it. But in recent times, the CFTC has withdrawn its permission for it to operate in the US.
***Social value***: My sense is that their research has usefully informed some OpenPhilanthropy decisions. Open Philanthropy donated around $80M in 2021 to the "Potential Risks from Advanced AI" cause area. A 5% improvement in that decision-making would imply a value of $4M/year.
_**Monetary value (valuation)**_: Insight Prediction gives it only an [11% chance of it surviving](https://insightprediction.com/m/18505/will-predictit-survive) ([a](http://web.archive.org/web/20221005064000/https://insightprediction.com/m/18505/will-predictit-survive)). My guess is that it was worth $5M to $50M when alive, implying a $0.5M to $5M valuation right now.
### Swift Centre For Applied Forecasting
_**Social value**_: PredictIt was fairly instrumental in the creation of a passionate if sometimes a bit cut-throat prediction market community. Its future value depends on its survival.
***What it is***: The [Swift Centre](https://www.swiftcentre.org/) ([a](http://web.archive.org/web/20221103141937/https://www.swiftcentre.org/)) publishes publicly available forecasts from accurate forecasters on topics of importance.
### Epoch
***Monetary value (grant flow):*** They have received [$2M from the FTX future fund regranting program](https://ftxfuturefund.org/our-regrants/?_search=Swift) ([a](https://web.archive.org/web/20221106220034/https://ftxfuturefund.org/our-regrants/?_search=Swift)).
_**What it is**_: [Epoch](https://epochai.org/) ([a](http://web.archive.org/web/20221013183722/https://epochai.org/)) is a research team investigating AI progress.
***Social value***: Their [forecasts](https://www.swiftcentre.org/) ([a](http://web.archive.org/web/20221103141937/https://www.swiftcentre.org/)) lead to better decisions. My sense is that the organization is still getting up to speed, and that when running at full operational capacity, they'll be producing many more predictions.
_**Monetary value (grant flow):**_ They have received around [$2M](https://www.openphilanthropy.org/grants/epoch-general-support/) ([a](http://web.archive.org/web/20220922190346/https://www.openphilanthropy.org/grants/epoch-general-support/)) from Open Philanthropy.
### Quantified Uncertainty Research Institute (QURI)
_**Social value**_: My sense is that their research has usefully informed some OpenPhilanthropy decisions. Open Philanthropy donated around $80M in 2021 to the “Potential Risks from Advanced AI” cause area. A 5% improvement in that decision-making would imply a value of $4M/year.
***What it is***: QURI is a "new initiative to advance forecasting and epistemics to improve the long-term future of humanity. We write research and make software." In particular, we are developing [Squiggle](https://squiggle-language.com/), a web-capable programming language for better estimation, and doing research around scalable forecasting and estimation.
### Swift Centre For Applied Forecasting
***Monetary value (grants flow)***: Overall we have received around $780k so far ([1](https://survivalandflourishing.fund/sff-2020-h1-recommendations) ([a](http://web.archive.org/web/20220703183731/https://survivalandflourishing.fund/sff-2020-h1-recommendations)), [2](https://survivalandflourishing.fund/sff-2022-h1-recommendations) ([a](http://web.archive.org/web/20220703184038/https://survivalandflourishing.fund/sff-2022-h1-recommendations)), [3](https://survivalandflourishing.fund/sff-2022-h1-recommendations) ([a](http://web.archive.org/web/20220703184038/https://survivalandflourishing.fund/sff-2022-h1-recommendations))), maybe a bit more.
_**What it is**_: The [Swift Centre](https://www.swiftcentre.org/) ([a](http://web.archive.org/web/20221103141937/https://www.swiftcentre.org/)) publishes publicly available forecasts from accurate forecasters on topics of importance.
***Social value***: Most of QURI's value is coming from the value of [Squiggle](https://www.squiggle-language.com/) ([a](http://web.archive.org/web/20221012160532/https://www.squiggle-language.com/)), which people are now starting to use for cost-effectiveness estimates, and from the value of [my own research](https://forum.effectivealtruism.org/users/nunosempere) ([a](http://web.archive.org/web/20221106162329/https://forum.effectivealtruism.org/users/nunosempere)), which might move the Effective Altruism movement more in the direction of using quantified estimates even for speculative stuff.
_**Monetary value (grant flow):**_ They have received [$2M from the FTX future fund regranting program](https://ftxfuturefund.org/our-regrants/?_search=Swift) ([a](https://web.archive.org/web/20221106220034/https://ftxfuturefund.org/our-regrants/?_search=Swift)).
***Note***: This is the organization that I work for.
_**Social value**_: Their [forecasts](https://www.swiftcentre.org/) ([a](http://web.archive.org/web/20221103141937/https://www.swiftcentre.org/)) lead to better decisions. My sense is that the organization is still getting up to speed, and that when running at full operational capacity, they’ll be producing many more predictions.
### Samotsvety Forecasting
### Quantified Uncertainty Research Institute (QURI)
***What it is***: [Samotsvety Forecasting](https://samotsvety.org/) ([a](http://web.archive.org/web/20221025004219/https://samotsvety.org/)) is a somewhat ad-hoc group of highly accurate forecasters of which I am a member.
_**What it is**_: QURI is a “new initiative to advance forecasting and epistemics to improve the long-term future of humanity. We write research and make software.” In particular, we are developing [Squiggle](https://squiggle-language.com/), a web-capable programming language for better estimation, and doing research around scalable forecasting and estimation.
***Monetary value***: My sense is that we could find some funder to sacrifice $200k to $5M if our group's existence depended on it. But I haven't actually tried that!
_**Monetary value (grants flow)**_: Overall we have received around $780k so far ([1](https://survivalandflourishing.fund/sff-2020-h1-recommendations) ([a](http://web.archive.org/web/20220703183731/https://survivalandflourishing.fund/sff-2020-h1-recommendations)), [2](https://survivalandflourishing.fund/sff-2022-h1-recommendations) ([a](http://web.archive.org/web/20220703184038/https://survivalandflourishing.fund/sff-2022-h1-recommendations)), [3](https://survivalandflourishing.fund/sff-2022-h1-recommendations) ([a](http://web.archive.org/web/20220703184038/https://survivalandflourishing.fund/sff-2022-h1-recommendations))), maybe a bit more.
***Social value:*** We've published a variety of public forecasts, and we find ourselves more and more to be a part of Effective Altruism's general epistemic infrastructure.
_**Social value**_: Most of QURI’s value is coming from the value of [Squiggle](https://www.squiggle-language.com/) ([a](http://web.archive.org/web/20221012160532/https://www.squiggle-language.com/)), which people are now starting to use for cost-effectiveness estimates, and from the value of [my own research](https://forum.effectivealtruism.org/users/nunosempere) ([a](http://web.archive.org/web/20221106162329/https://forum.effectivealtruism.org/users/nunosempere)), which might move the Effective Altruism movement more in the direction of using quantified estimates even for speculative stuff.
### Czech Priorities
_**Note**_: This is the organization that I work for.
***What it is***: [Czech Priorities](https://www.ceskepriority.cz/) ([a](http://web.archive.org/web/20221102103806/https://www.ceskepriority.cz/)) is a Czech group working to advance forecasting within the Czech Republic.
### Samotsvety Forecasting
***Monetary value:*** I couldn't find grantmaking information from a quick online search.
_**What it is**_: [Samotsvety Forecasting](https://samotsvety.org/) ([a](http://web.archive.org/web/20221025004219/https://samotsvety.org/)) is a somewhat ad-hoc group of highly accurate forecasters of which I am a member.
***Social value***: It's possible that they could find generalizable lessons about using forecasting to influence policy and improve decision-making that could then be extended to other countries. But this is not certain.
_**Monetary value**_: My sense is that we could find some funder to sacrifice $200k to $5M if our group’s existence depended on it. But I haven’t actually tried that!
### Hedgehog Markets
_**Social value:**_ We’ve published a variety of public forecasts, and we find ourselves more and more to be a part of Effective Altruism’s general epistemic infrastructure.
***What it is***: [Hedghog Markets](https://hedgehog.markets/) ([a](http://web.archive.org/web/20221010202946/https://hedgehog.markets/)) is a crypto-prediction market
### Czech Priorities
***Monetary value***: They raised [$
3.5M in funding](https://www.crunchbase.com/organization/hedgehog-markets) back in the crypto bull days.
_**What it is**_: [Czech Priorities](https://www.ceskepriority.cz/) ([a](http://web.archive.org/web/20221102103806/https://www.ceskepriority.cz/)) is a Czech group working to advance forecasting within the Czech Republic.
***Social value***: They have been particularly innovative around no-loss markets, and peer-to-peer markets. But value depends on future usage volume.
_**Monetary value:**_ I couldn’t find grantmaking information from a quick online search.
### INFER
_**Social value**_: It’s possible that they could find generalizable lessons about using forecasting to influence policy and improve decision-making that could then be extended to other countries. But this is not certain.
***What it is***: INFER is a prediction market platform which seeks to influence US government policy around transformative technologies.
### Hedgehog Markets
***Monetary value***: INFER is hosted at [ARLIS](https://www.arlis.umd.edu/home) ([a](http://web.archive.org/web/20221013111200/https://www.arlis.umd.edu/home)), at the University of Maryland, which received an ~[$8M grant from Open Philanthropy specifically for forecasting](https://www.openphilanthropy.org/grants/applied-research-laboratory-for-intelligence-and-security-forecasting-platforms/) ([a](http://web.archive.org/web/20220808081108/https://www.openphilanthropy.org/grants/applied-research-laboratory-for-intelligence-and-security-forecasting-platforms/)) (over two years, over two platforms), which I find very confusing because I don't think they have deployed much of that capital yet.
_**What it is**_: [Hedghog Markets](https://hedgehog.markets/) ([a](http://web.archive.org/web/20221010202946/https://hedgehog.markets/)) is a crypto-prediction market
***Social value***: INFER aims to influence decisions by the US government around technology policy. To the extent that this is the case, it could be valuable indeed, but this is hard to evaluate from the outside.
_**Monetary value**_: They raised [$3.5M in funding](https://www.crunchbase.com/organization/hedgehog-markets) back in the crypto bull days.
## Smaller forecasting entities
_**Social value**_: They have been particularly innovative around no-loss markets, and peer-to-peer markets. But value depends on future usage volume.
### Nathan Young
### INFER
Nathan Young is a smallish forecasting influencer followed by 5k people on Twitter. He recently got [$182k from the FTX Future Fund regranting program](https://ftxfuturefund.org/our-grants/?_search=Nathan%20Young) ([a](https://web.archive.org/web/20221106220135/https://ftxfuturefund.org/our-grants/?_search=Nathan%20Young)) to build a platform for forecasting question generation, an alpha version of which can be seen [here](https://doubtful.app/) ([a](http://web.archive.org/web/20221031061542/https://doubtful.app/)).
_**What it is**_: INFER is a forecasting platform which seeks to influence US government policy around transformative technologies.
### Sage
_**Monetary value**_: INFER is hosted at [ARLIS](https://www.arlis.umd.edu/home) ([a](http://web.archive.org/web/20221013111200/https://www.arlis.umd.edu/home)), at the University of Maryland, which received an ~[$8M grant from Open Philanthropy specifically for forecasting](https://www.openphilanthropy.org/grants/applied-research-laboratory-for-intelligence-and-security-forecasting-platforms/) ([a](http://web.archive.org/web/20220808081108/https://www.openphilanthropy.org/grants/applied-research-laboratory-for-intelligence-and-security-forecasting-platforms/)) (over two years, over two platforms), which I find very confusing because I don’t think they have deployed much of that capital yet.
Sage is an organization dedicated to forecasting tooling and research. It got $700k from the FTX Future Fund, and has built the tools on [quantifiedintuitions.org](https://www.quantifiedintuitions.org/pastcasting) ([a](http://web.archive.org/web/20221013134558/https://www.quantifiedintuitions.org/pastcasting)). But it hasn't deployed most of its capital yet.
_**Social value**_: INFER aims to influence decisions by the US government around technology policy. To the extent that this is the case, it could be valuable indeed, but this is hard to evaluate from the outside.
### Global Guessing
## Smaller forecasting entities
[Global Guessing](https://globalguessing.com/) ([a](http://web.archive.org/web/20221031010912/https://globalguessing.com/)) is a forecasting site which made and gathered forecasts early on the Russian invasion of Ukraine, but which seems pretty dormant since then. They have received [~$330k from the FTX Future Fund.](https://ftxfuturefund.org/our-regrants/?_search=Global%20Guessing) ([a](https://web.archive.org/web/20221106220157/https://ftxfuturefund.org/our-regrants/?_search=Global%20Guessing))
### Nathan Young
### Social Science Prediction Platform
[Nathan Young](https://twitter.com/NathanpmYoung) is a smallish forecasting influencer followed by 6k people on Twitter. He recently got [$182k from the FTX Future Fund regranting program](https://ftxfuturefund.org/our-grants/?_search=Nathan%20Young) ([a](https://web.archive.org/web/20221106220135/https://ftxfuturefund.org/our-grants/?_search=Nathan%20Young)) to build a platform for forecasting question generation, an alpha version of which can be seen [here](https://doubtful.app/) ([a](http://web.archive.org/web/20221031061542/https://doubtful.app/)).
The [Social Science Prediction Platform](https://socialscienceprediction.org/) ([a](http://web.archive.org/web/20221013155854/https://socialscienceprediction.org/)) collects forecasts from academics on upcoming papers. This can have a range of public benefits, such as estimates of how surprising results in the social sciences are. Personally I'd be hopeful about the ability to construct a more legible or objective Bayesian prior. I could quickly find that they have received [$346k](https://survivalandflourishing.fund/sff-2022-h1-recommendations) ([a](http://web.archive.org/web/20220703184038/https://survivalandflourishing.fund/sff-2022-h1-recommendations)) from the Survival and Flourishing Fund.
### Sage
### Augur
Sage is an organization dedicated to forecasting tooling and research. It got $700k from the FTX Future Fund, and has built the tools on [quantifiedintuitions.org](https://www.quantifiedintuitions.org/pastcasting) ([a](http://web.archive.org/web/20221013134558/https://www.quantifiedintuitions.org/pastcasting)). But it hasn’t deployed most of its capital yet.
[Augur](https://augur.net/) was a pioneering prediction market way back when, but isn't up to much these days. Somehow the implied [market cap](https://coinmarketcap.com/currencies/augur/) ([a](http://web.archive.org/web/20220619100141/https://coinmarketcap.com/currencies/augur/)) of their coin is still $74M (## ?!?).
### Global Guessing
### Confido
[Global Guessing](https://globalguessing.com/) ([a](http://web.archive.org/web/20221031010912/https://globalguessing.com/)) is a forecasting site which made and gathered forecasts early on the Russian invasion of Ukraine, but which seems pretty dormant since then. They have received [~$330k from the FTX Future Fund.](https://ftxfuturefund.org/our-regrants/?_search=Global%20Guessing) ([a](https://web.archive.org/web/20221106220157/https://ftxfuturefund.org/our-regrants/?_search=Global%20Guessing))
[Confido](https://confido.tools/) ([a](http://web.archive.org/web/20220711161543/https://confido.tools/)) is a three-person strong Czech organization which develops forecasting tooling. They received $190k from the [FTX Future Fund regranting program](https://ftxfuturefund.org/our-regrants/?_search=Confido) ([a](https://web.archive.org/web/20221106220218/https://ftxfuturefund.org/our-regrants/?_search=Confido)).
### Social Science Prediction Platform
### Hypermind
The [Social Science Prediction Platform](https://socialscienceprediction.org/) ([a](http://web.archive.org/web/20221013155854/https://socialscienceprediction.org/)) collects forecasts from academics on upcoming papers. This can have a range of public benefits, such as estimates of how surprising results in the social sciences are. Personally I’d be hopeful about the ability to construct a more legible or objective Bayesian prior. I could quickly find that they have received [$346k](https://survivalandflourishing.fund/sff-2022-h1-recommendations) ([a](http://web.archive.org/web/20220703184038/https://survivalandflourishing.fund/sff-2022-h1-recommendations)) from the Survival and Flourishing Fund, and an additional $492k from the FTX Future Fund's [regranting program](https://ftxfuturefund.org/our-regrants/?_area_of_interest=epistemic-institutions) ([a](https://web.archive.org/web/20221109161427/https://ftxfuturefund.org/our-regrants/?_area_of_interest=epistemic-institutions))
[Hypermind ](https://www.hypermind.com/en/) ([a](http://web.archive.org/web/20221015015526/https://www.hypermind.com/en/))is a French forecasting organization which hosts some low-stakes markets with play money but real-money rewards.
### Augur
### Replication Markets
[Augur](https://augur.net/) was a pioneering prediction market way back when, but isn’t up to much these days. Somehow the implied [market cap](https://coinmarketcap.com/currencies/augur/) ([a](http://web.archive.org/web/20220619100141/https://coinmarketcap.com/currencies/augur/)) of their coin is still ~$60M (?!?).
[Replication Markets](https://replicationmarkets.com/) ([a](http://web.archive.org/web/20220930085636/https://replicationmarkets.com/)) was a super-interesting experiment in having forecasters predict the outcome of replications. It has now ended, and it's uncertain whether the creators will follow up with something as interesting. They had ~$150k in forecaster rewards.
### Confido
### PredictionBook
[Confido](https://confido.tools/) ([a](http://web.archive.org/web/20220711161543/https://confido.tools/)) is a three-person strong Czech organization which develops forecasting tooling. They received $190k from the [FTX Future Fund regranting program](https://ftxfuturefund.org/our-regrants/?_search=Confido) ([a](https://web.archive.org/web/20221106220218/https://ftxfuturefund.org/our-regrants/?_search=Confido)).
[PredictionBook ](https://predictionbook.com/) ([a](http://web.archive.org/web/20221027154039/https://predictionbook.com/))is an old, old site used by people to keep track of their probabilities, with no rewards involved. Gwern still [uses it](https://predictionbook.com/users/gwern) ([a](http://web.archive.org/web/20221019042049/https://predictionbook.com/users/gwern)).
### Hypermind
### Gnosis
[Hypermind](https://www.hypermind.com/en/) ([a](http://web.archive.org/web/20221015015526/https://www.hypermind.com/en/))is a French forecasting organization which hosts some low-stakes markets with play money but real-money rewards.
Gnosis is much like Augur, but less successful. They also have a [GnosisDAO](https://gnosis.io/gnosisdao/) ([a](http://web.archive.org/web/20220825162029/https://gnosis.io/gnosisdao/)). The implied market cap of their token is worth [$330M](https://gnosis.io/gnosisdao/) ([a](http://web.archive.org/web/20220825162029/https://gnosis.io/gnosisdao/)), based on the [Gnosis safe](https://gnosis-safe.io/) ([a](http://web.archive.org/web/20221101042815/https://gnosis-safe.io/)). But it doesn't do much forecasting these days.
### Replication Markets
## Thoughts on this ranking
[Replication Markets](https://replicationmarkets.com/) ([a](http://web.archive.org/web/20220930085636/https://replicationmarkets.com/)) was a super-interesting experiment in having forecasters predict the outcome of replications. It has now ended, and it’s uncertain whether the creators will follow up with something as interesting. They had ~$150k in forecaster rewards.
### On the money flows
### PredictionBook
Money flows are interesting, but flawed as a measure of value. The spending rate or VC money raised does not correspond to the value a project creates. The high valuations for crypto projects could be mostly illusory, and for the case of Gnosis and Augur, they probably are.
[PredictionBook](https://predictionbook.com/) ([a](http://web.archive.org/web/20221027154039/https://predictionbook.com/))is an old, old site used by people to keep track of their probabilities, with no rewards involved. Gwern still [uses it](https://predictionbook.com/users/gwern) ([a](http://web.archive.org/web/20221019042049/https://predictionbook.com/users/gwern)).
Still, a monetary ranking has some advantages
### Gnosis
- There may be some ambiguity to it, but it allows one to differentiate between \<$500k projects, $500k to $5M projects, and $5M to $50M projects.
- It has an honesty to it, in that money is a somewhat hard-to-fake signal
- It is a fast proxy to estimate
Gnosis is much like Augur, but less successful. They also have a [GnosisDAO](https://gnosis.io/gnosisdao/) ([a](http://web.archive.org/web/20220825162029/https://gnosis.io/gnosisdao/)). The implied market cap of their token is worth [$230M](https://gnosis.io/gnosisdao/) ([a](http://web.archive.org/web/20220825162029/https://gnosis.io/gnosisdao/)), based on the [Gnosis safe](https://gnosis-safe.io/) ([a](http://web.archive.org/web/20221101042815/https://gnosis-safe.io/)). But it doesn’t do much forecasting these days.
### On the estimate of social value
## Thoughts on this ranking
The estimate of social value isn't a quantified estimate, but rather a few lines of observations, and maybe a path to impact. I think I could translate this into a numerical relative value scale, but it would take a fair amount of time.
### On the money flows
### On the different pathways to impact
Money flows are interesting, but flawed as a measure of value. The spending rate or VC money raised does not correspond to the value a project creates. The high valuations for crypto projects could be mostly illusory, and for the case of Gnosis and Augur, they probably are.
I'm seeing mainly three pathways of impact:
Still, a monetary ranking has some advantages
1. Providing consumer value, e.g., betting markets or hedging utilities that people want to use
2. Providing informative markets about events of interest, which can apply pressure to decision-makers inside governments and corporations.
3. Providing tooling that people making decisions can make and incorporate as part of their decision-making
* There may be some ambiguity to it, but it allows one to differentiate between &lt;$500k projects, $500k to $5M projects, and $5M to $50M projects.
* It has an honesty to it, in that money is a somewhat hard-to-fake signal
* It is a fast proxy to estimate
I'm mostly bullish on the third option, but this deserves more elaboration.
### On the estimate of social value
The estimate of social value isn’t a quantified estimate, but rather a few lines of observations, and maybe a path to impact. I think I could translate this into a numerical relative value scale, but it would take a fair amount of time.
### On the different pathways to impact
I’m seeing mainly three pathways of impact for forecasting projects:
1. Providing consumer value, e.g., betting markets or hedging utilities that people want to use
2. Providing informative markets about events of interest, which can apply pressure to decision-makers inside governments and corporations to make better decisions
3. Providing tooling that people making decisions can make and incorporate as part of their decision-making
I’m mostly bullish on the third option, but this deserves more elaboration.
### On how this estimate could be improved
This estimate could be improved by having numerical estiamates of impact. This would allow for comparisons around efficiency &c. It could also be presented better, i.e., in a table, or more creatively, in a map.
### Significance given FTX's current troubles
Given FTX's current troubles, it's very possible that there will be less money floating around for altruistic forecasting projects. At the same time, some of the projects FTX has donated to may  prove successful, and other funders may want to step in.
<p><section id="isso-thread">
<noscript>Javascript needs to be activated to view comments.</noscript>
</section></p>
</article>
</section></p>