diff --git a/README.md b/README.md index 2d04538..0bbc2e3 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,8 @@ ## What this is -https://metaforecast.org is a search engine for probabilities from various prediction markes and forecasting platforms (try searching "Trump", "China" or "Semiconductors"). +[Metaforecast](https://metaforecast.org) is a search engine for probabilities from various prediction markes and forecasting platforms. Try searching "Trump", "China" or "Semiconductors". -This repository includes a source code for the website, as well as a set of libraries that fetches probabilities/forecasts. - -We also provide a public database, which can be accessed with a script similar to [this one](./src/backend/manual/manualDownload.js). - -I also created a search engine using Elicit's IDE, which uses GPT-3 to deliver vastly superior semantic search (as opposed to fuzzy word matching). If you have access to the Elicit IDE, you can use the action "Search Metaforecast database". However, I'm not currently updating it regularly. +This repository includes the source code for both the website and the library that fetches forecasts needed to replace them. We also aim to provide tooling to integrate metaforecast with other services. [![](./public/screenshot.png)](https://metaforecast.org) @@ -30,7 +26,7 @@ See [./docs/configuration.md](./docs/configuration.md) for details. ### 3. Actually run -`npm run cli` starts a local CLI which presents the user with choices; if you would like to skip each step, use the option number instead, e.g., `npm run start 14`. +`npm run cli` starts a local CLI which presents the user with choices; if you would like to skip each step, use the option number instead, e.g., `npm run cli 14`. `npm run next-dev` starts a Next.js dev server with the website on `http://localhost:3000`. @@ -43,6 +39,19 @@ $ npm install $ node src/backend/manual/manualDownload.js ``` +## Integrations + +Metaforecast has been integrated into: + +- Twitter, using our [@metaforecast](https://twitter.com/metaforecast) bot +- [Global Guessing](https://globalguessing.com/russia-ukraine-forecasts/), which integrates our dashboards +- [Fletcher](https://fletcher.fun/), a popular Discord bot. You can invoke metaforecast with `!metaforecast search-term` +- [Elicit](https://elicit.org/), which uses GPT-3 to deliver vastly superior semantic search (as opposed to fuzzy word matching). If you have access to the Elicit IDE, you can use the action "Search Metaforecast database. This is not being updated regularly. + +We also provide a public database, which can be accessed with a script similar to [this one](./src/backend/manual/manualDownload.js). We are also open to integrating our Algolia search instance with other trusted services (in addition to Fletcher.) + +In general, if you want to integrate metaforecast into your service, we want to hear from you. + ## Code layout - frontend code is in [src/pages/](./src/pages/), [src/web/](./src/web/) and in a few other places which are required by Next.js (e.g. root-level configs in postcss.config.js and tailwind.config.js) @@ -62,11 +71,10 @@ Overall, the services which we use are: - Algolia for search - Netlify for website deployment -- DigitalOcean for background jobs, e.g. fetching new forecasts +- Heroku for background jobs, e.g. fetching new forecasts - Postgres on DigitalOcean for database ## Various notes - Commits follow [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summary) -- Right now, I'm fetching only a couple of common properties, such as the title, url, platform, whether a question is binary (yes/no), its percentage, and the number of forecasts. -- For elicit and metaculus, this library currently filters questions with <10 predictions. +- For elicit and metaculus, this library currently filters out questions with <10 predictions. diff --git a/package-lock.json b/package-lock.json index c8029f4..546056b 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "metaforecast", - "version": "1.0.0", + "version": "2.0.0", "lockfileVersion": 2, "requires": true, "packages": { "": { "name": "metaforecast", - "version": "1.0.0", + "version": "2.0.0", "license": "MIT", "dependencies": { "@tailwindcss/forms": "^0.4.0", @@ -33,6 +33,7 @@ "json2csv": "^5.0.5", "multiselect-react-dropdown": "^2.0.17", "next": "12", + "next-plausible": "^3.1.6", "nprogress": "^0.2.0", "open": "^7.3.1", "papaparse": "^5.3.0", @@ -31952,6 +31953,19 @@ } } }, + "node_modules/next-plausible": { + "version": "3.1.6", + "resolved": "https://registry.npmjs.org/next-plausible/-/next-plausible-3.1.6.tgz", + "integrity": "sha512-9L1l3f59122IBNjVDqZEE2y5kA/a8rk3xYqWWfHH9LLxtWw6hqiL43RQGT7xYjHofGdn+RX3WTVRGIv0Iq/wKA==", + "funding": { + "url": "https://github.com/4lejandrito/next-plausible?sponsor=1" + }, + "peerDependencies": { + "next": "^11.1.0 || ^12.0.0", + "react": "^16.8.0 || ^17.0.0", + "react-dom": "^16.8.0 || ^17.0.0" + } + }, "node_modules/next/node_modules/postcss": { "version": "8.4.5", "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.4.5.tgz", @@ -58677,6 +58691,12 @@ } } }, + "next-plausible": { + "version": "3.1.6", + "resolved": "https://registry.npmjs.org/next-plausible/-/next-plausible-3.1.6.tgz", + "integrity": "sha512-9L1l3f59122IBNjVDqZEE2y5kA/a8rk3xYqWWfHH9LLxtWw6hqiL43RQGT7xYjHofGdn+RX3WTVRGIv0Iq/wKA==", + "requires": {} + }, "node-abi": { "version": "3.8.0", "resolved": "https://registry.npmjs.org/node-abi/-/node-abi-3.8.0.tgz", diff --git a/package.json b/package.json index a49ef52..796d65d 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "metaforecast", - "version": "1.0.0", + "version": "2.0.0", "description": "Get forecasts from various platforms", "repository": { "type": "git", @@ -51,6 +51,7 @@ "json2csv": "^5.0.5", "multiselect-react-dropdown": "^2.0.17", "next": "12", + "next-plausible": "^3.1.6", "nprogress": "^0.2.0", "open": "^7.3.1", "papaparse": "^5.3.0", diff --git a/src/backend/manual/pullSuperforecastsManually.sh b/src/backend/manual/pullSuperforecastsManually.sh index 885d2a0..c152cb9 100755 --- a/src/backend/manual/pullSuperforecastsManually.sh +++ b/src/backend/manual/pullSuperforecastsManually.sh @@ -1,4 +1,4 @@ #!/bin/bash -cd /home/loki/Documents/core/software/fresh/js/metaforecast/metaforecast-backend -date > ./notes/last-superforecast-pull.txt -/home/loki/.nvm/versions/node/v16.8.0/bin/node ./src/manual/pullSuperforecastsManually.js >> ./notes/last-superforecast-pull.txt \ No newline at end of file + +cd /home/loki/Documents/core/software/fresh/js/metaforecast/metaforecast-monorepo +/home/loki/.nvm/versions/node/v17.5.0/bin/npm run cli 3 > ../last-superforecast-fetch.txt diff --git a/src/pages/_app.tsx b/src/pages/_app.tsx index 86e1028..7afae05 100644 --- a/src/pages/_app.tsx +++ b/src/pages/_app.tsx @@ -1,8 +1,10 @@ -import 'nprogress/nprogress.css'; -import '../styles/main.css'; +import "nprogress/nprogress.css"; +import "../styles/main.css"; -import Router from 'next/router'; -import NProgress from 'nprogress'; +import Router from "next/router"; +import NProgress from "nprogress"; + +import PlausibleProvider from "next-plausible"; Router.events.on("routeChangeStart", (as, { shallow }) => { console.log(shallow); @@ -14,7 +16,11 @@ Router.events.on("routeChangeComplete", () => NProgress.done()); Router.events.on("routeChangeError", () => NProgress.done()); function MyApp({ Component, pageProps }) { - return ; + return ( + + + + ); } export default MyApp; diff --git a/src/pages/about.tsx b/src/pages/about.tsx index bbd68cb..4590e84 100644 --- a/src/pages/about.tsx +++ b/src/pages/about.tsx @@ -1,16 +1,16 @@ -import React from 'react'; -import ReactMarkdown from 'react-markdown'; -import gfm from 'remark-gfm'; +import React from "react"; +import ReactMarkdown from "react-markdown"; +import gfm from "remark-gfm"; -import Layout from '../web/display/layout'; +import Layout from "../web/display/layout"; let readmeMarkdownText = `# About -This is a search engine for probabilities. Given a query, it searches for relevant questions in various prediction markets and forecasting platforms (namely Betfair, FantasySCOTUS, Foretold, Good Judgment, Good Judgment Open, Guesstimate, GiveWell & OpenPhilanthropy blogposts as of early 2021, Kalshi, Infer, Ladbrokes, Manifold Markets, Metaculus, Polymarket, PredictIt, Smarkets, William Hill, and Michael Aird's database of existential risk estimates). For example, try searching for "China", "North Korea", "Semiconductors", "COVID", "Trump", or "X-risk". +This webpage is a search engine for probabilities. Given a query, it searches for relevant questions in various prediction markets and forecasting platforms. For example, try searching for "China", "North Korea", "Semiconductors", "COVID", "Trump", or "X-risk". In addition to search, we also provide various [tools](http://localhost:3000/tools). -You can also call the metaforecast database from within Twitter by mentioning @metaforecast, [like so](https://twitter.com/NunoSempere/status/1433160907308294144). +We are very interested in integrating Metaforecast with other services, and the whole thing is [open source](https://github.com/QURIresearch/metaforecast). So far, Metaforecast has been integrated with [Twitter](https://twitter.com/NunoSempere/status/1433160907308294144), [Fletcher](https://fletcher.fun/), [GlobalGuessing](https://globalguessing.com/russia-ukraine-forecasts/) and previously [Elicit](https://elicit.org/). -You can read a longer writeup with thoughts and motivations [here](https://forum.effectivealtruism.org/posts/tEo5oXeSNcB3sYr8m/introducing-metaforecast-a-forecast-aggregator-and-search), and an update thereto [here](https://www.lesswrong.com/posts/5hugQzRhdGYc6ParJ/metaforecast-update-better-search-capture-functionality-more). You can read the code for the back-end used to fetch probabilities [here](https://github.com/QURIresearch/metaforecast)— including instructions for how to download the underlying dataset, or view the source for the front-end [here](https://github.com/QURIresearch/metaforecast-frontend). +You can read a longer writeup with thoughts and motivations [here](https://forum.effectivealtruism.org/posts/tEo5oXeSNcB3sYr8m/introducing-metaforecast-a-forecast-aggregator-and-search), and an update thereto [here](https://www.lesswrong.com/posts/5hugQzRhdGYc6ParJ/metaforecast-update-better-search-capture-functionality-more). ## Advanced search If your initial search doesn't succeed, you might want to try tinkering with the advanced search. In particular, try increasing or decreasing the stars threshold, or changing the number of search results shown.