tweak: Add license, some small tweaks.

upgrade: safe dependencies
feat: upgrade dependencies part 1.
2023-01-05 18:24:26 +01:00 · 2022-11-29 09:10:30 +00:00 · 2022-11-29 09:00:31 +00:00 · 2022-11-26 13:27:00 +00:00 · 2022-11-17 21:58:15 +00:00 · 2022-11-17 20:16:24 +00:00
43 changed files with 10318 additions and 70315 deletions
--- a/.gitignore
+++ b/.gitignore
@ -31,6 +31,7 @@ yarn-error.log*

 # yarn vs npm conflict
 package-lock.json ## use yarn.lock instead
+# yarn.lock

 # Local Netlify folder
 .netlify
--- a/LICENSE.md
+++ b/LICENSE.md
@ -0,0 +1,7 @@
+Copyright 2023 Quantified Uncertainty Research Institute.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
--- a/README.md
+++ b/README.md
@ -11,8 +11,8 @@ This repository includes the source code for both the website and the library th
 ### 1. Download this repository

 ```
-$ git clone https://github.com/QURIresearch/metaforecast
-$ cd metaforecasts
+$ git clone https://github.com/quantified-uncertainty/metaforecast
+$ cd metaforecast
 $ npm install
 ```

@ -26,17 +26,28 @@ See [./docs/configuration.md](./docs/configuration.md) for details.

 ### 3. Actually run

-`npm run cli` starts a local CLI which presents the user with choices. If you would like to skip that step, use the option name instead, e.g., `npm run cli wildeford`.
+After installing and building (`npm run build`) the application, `npm run cli` starts a local CLI which presents the user with choices. If you would like to skip that step, use the option name instead, e.g., `npm run cli wildeford`.

 ![](./public/screenshot-cli.png)

 `npm run next-dev` starts a Next.js dev server with the website on `http://localhost:3000`.

+So overall this would look like
+
+```
+$ git clone https://github.com/quantified-uncertainty/metaforecast
+$ cd metaforecast
+$ npm install
+$ npm run build
+$ npm run cli
+$ npm run next-dev
+```
+
 ### 4. Example: download the metaforecasts database

 ```
-$ git clone https://github.com/QURIresearch/metaforecast
-$ cd metaforecasts
+$ git clone https://github.com/quantified-uncertainty/metaforecast
+$ cd metaforecast
 $ npm install
 $ node src/backend/manual/manualDownload.js
 ```
@ -78,5 +89,22 @@ Overall, the services which we use are:

 ## Various notes

+- This repository is released under the [MIT license](https://opensource.org/licenses/MIT). See `LICENSE.md`
 - Commits follow [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summary)
 - For elicit and metaculus, this library currently filters out questions with <10 predictions.
+- The database is updated once a day, at 3:00 AM UTC, with the command `ts-node -T src/backend/flow/doEverythingForScheduler.ts`. The frontpage is updated after that, at 6:00 AM UTC with the command `ts-node -T src/backend/index.ts frontpage`. It's possible that either of these two operations makes the webpage briefly go down.
+
+## To do
+
+- [x] Update Metaculus and Manifold Markets fetchers
+- [x] Add markets from [Insight Prediction](https://insightprediction.com/).
+- [ ] Use <https://news.manifold.markets/p/above-the-fold-midterms-special> to update stars calculation for Manifold.
+- [ ] Add a few more snippets, with fetching individual questions, questions with histories, questions added within the last 24h to the /contrib folder (good first issue)
+- [ ] Refactor code so that users can capture and push the question history chart to imgur (good first issue)
+- [ ] Upgrade to [React 18](https://reactjs.org/blog/2022/03/08/react-18-upgrade-guide.html). This will require dealing with the workaround we used for [this issue](https://github.com/vercel/next.js/issues/36019#issuecomment-1103266481)
+- [ ] Add database of resolutions
+- [ ] Allow users to embed predictions in the EA Forum/LessWrong (in progress)
+- [ ] Find a long-term mantainer for this project
+- [ ] Allow users to record their own predictions
+- [ ] Release snapshots (I think @niplav is working on this)
+- [ ] ...
--- a/package-lock.json
+++ b/package-lock.json
--- a/package.json
+++ b/package.json
@ -24,93 +24,101 @@
    "build": "prisma generate && next build",
    "next-start": "next start",
    "next-export": "next export",
-    "dbshell": ". .env && psql $DIGITALOCEAN_POSTGRES"
+    "dbshell": ". .env && psql $DIGITALOCEAN_POSTGRES",
+    "upgrade-interactive": "yarn upgrade-interactive --latest"
  },
  "dependencies": {
-    "@floating-ui/react-dom": "^0.7.0",
-    "@graphql-yoga/node": "^2.1.0",
-    "@pothos/core": "^3.5.1",
-    "@pothos/plugin-prisma": "^3.4.0",
-    "@pothos/plugin-relay": "^3.10.0",
-    "@prisma/client": "^3.11.1",
-    "@quri/squiggle-lang": "^0.2.8",
-    "@tailwindcss/forms": "^0.4.0",
-    "@tailwindcss/typography": "^0.5.1",
-    "@types/chroma-js": "^2.1.3",
+    "@floating-ui/react-dom": "^0.7.2",
+    "@graphql-yoga/plugin-response-cache": "^1.1.0",
+    "@pothos/core": "^3.22.8",
+    "@pothos/plugin-prisma": "^3.35.6",
+    "@pothos/plugin-relay": "^3.28.6",
+    "@prisma/client": "^3.15.2",
+    "@quri/squiggle-lang": "^0.5.1",
+    "@tailwindcss/forms": "^0.4.1",
+    "@tailwindcss/typography": "^0.5.7",
+    "@types/chroma-js": "^2.1.4",
    "@types/dom-to-image": "^2.6.4",
-    "@types/google-spreadsheet": "^3.2.1",
-    "@types/jsdom": "^16.2.14",
+    "@types/google-spreadsheet": "^3.3.0",
+    "@types/jsdom": "^16.2.15",
    "@types/nprogress": "^0.2.0",
-    "@types/react": "^17.0.39",
-    "@types/react-copy-to-clipboard": "^5.0.2",
+    "@types/react": "<18.0.0",
+    "@types/react-copy-to-clipboard": "^5.0.4",
    "@types/textversionjs": "^1.1.1",
    "@types/tunnel": "^0.0.3",
-    "airtable": "^0.11.1",
-    "algoliasearch": "^4.10.3",
-    "autoprefixer": "^10.1.0",
-    "axios": "^0.25.0",
+    "airtable": "^0.11.5",
+    "ajv": "^8.11.0",
+    "algoliasearch": "^4.14.2",
+    "autoprefixer": "10.4.5",
+    "axios": "^1.2.0",
    "chroma-js": "^2.4.2",
    "critters": "^0.0.16",
-    "date-fns": "^2.28.0",
+    "date-fns": "^2.29.3",
    "dom-to-image": "^2.6.0",
-    "dotenv": "^16.0.0",
+    "dotenv": "^16.0.3",
    "fetch": "^1.1.0",
    "fs": "^0.0.1-security",
-    "fuse.js": "^6.4.6",
-    "google-spreadsheet": "^3.1.15",
-    "graphql": "^16.3.0",
-    "graphql-request": "^4.0.0",
-    "html-to-image": "^1.7.0",
+    "fuse.js": "^6.6.2",
+    "google-spreadsheet": "^3.3.0",
+    "graphql": "^16.6.0",
+    "graphql-request": "^5.0.0",
+    "graphql-yoga": "^3.0.0-next.10",
+    "html-to-image": "^1.10.8",
    "https": "^1.0.0",
    "isomorphic-fetch": "^3.0.0",
    "jsdom": "^19.0.0",
-    "json2csv": "^5.0.5",
-    "multiselect-react-dropdown": "^2.0.17",
-    "next": "12",
-    "next-plausible": "^3.1.6",
-    "next-urql": "^3.3.2",
+    "json2csv": "^5.0.7",
+    "multiselect-react-dropdown": "^2.0.25",
+    "next": "^12.3.1",
+    "next-plausible": "^3.6.3",
+    "next-urql": "^3.3.3",
    "nprogress": "^0.2.0",
-    "open": "^7.3.1",
-    "papaparse": "^5.3.0",
-    "pg": "^8.7.3",
-    "postcss": "^8.2.1",
+    "open": "^7.4.2",
+    "papaparse": "^5.3.2",
+    "pg": "^8.8.0",
+    "postcss": "^8.4.18",
    "postcss-flexbugs-fixes": "^5.0.2",
-    "postcss-preset-env": "^7.3.2",
-    "prisma": "^3.11.1",
+    "postcss-preset-env": "^7.8.2",
+    "prisma": "^3.15.2",
    "query-string": "^7.1.1",
+    "re-resizable": "^6.9.9",
    "react": "^17.0.2",
    "react-component-export-image": "^1.0.6",
-    "react-compound-slider": "^3.3.1",
-    "react-copy-to-clipboard": "^5.0.3",
+    "react-compound-slider": "^3.4.0",
+    "react-copy-to-clipboard": "^5.1.0",
    "react-dom": "^17.0.2",
-    "react-dropdown": "^1.9.2",
-    "react-hook-form": "^7.27.0",
-    "react-icons": "^4.2.0",
-    "react-is": "^18.0.0",
-    "react-markdown": "^8.0.0",
+    "react-dropdown": "^1.11.0",
+    "react-hook-form": "^7.38.0",
+    "react-icons": "^4.6.0",
+    "react-is": "^18.2.0",
+    "react-markdown": "^8.0.3",
    "react-safe": "^1.3.0",
-    "react-select": "^5.2.2",
+    "react-select": "^5.5.4",
    "remark-gfm": "^3.0.1",
-    "tabletojson": "^2.0.4",
-    "tailwindcss": "^3.0.22",
+    "tabletojson": "^2.0.7",
+    "tailwindcss": "^3.2.0",
    "textversionjs": "^1.1.3",
-    "ts-node": "^10.7.0",
+    "ts-node": "^10.9.1",
    "tunnel": "^0.0.6",
-    "urql": "^2.2.0",
-    "urql-custom-scalars-exchange": "^0.1.5",
-    "victory": "^36.3.2"
+    "urql": "^2.2.3",
+    "urql-custom-scalars-exchange": "^0.1.6",
+    "victory": "^36.6.8"
+  },
+  "resolutions": {
+    "@types/react": "<18.0.0"
  },
  "devDependencies": {
-    "@graphql-codegen/cli": "^2.6.2",
-    "@graphql-codegen/introspection": "^2.1.1",
-    "@graphql-codegen/near-operation-file-preset": "^2.2.9",
-    "@graphql-codegen/schema-ast": "^2.4.1",
-    "@graphql-codegen/typed-document-node": "^2.2.8",
-    "@graphql-codegen/typescript": "^2.4.8",
-    "@graphql-codegen/typescript-operations": "^2.3.5",
-    "@netlify/plugin-nextjs": "^4.2.4",
-    "@svgr/cli": "^6.2.1",
+    "@graphql-codegen/cli": "^2.13.7",
+    "@graphql-codegen/introspection": "^2.2.1",
+    "@graphql-codegen/near-operation-file-preset": "^2.4.3",
+    "@graphql-codegen/schema-ast": "^2.5.1",
+    "@graphql-codegen/typed-document-node": "^2.3.5",
+    "@graphql-codegen/typescript": "^2.7.5",
+    "@graphql-codegen/typescript-operations": "^2.5.5",
+    "@svgr/cli": "^6.5.0",
    "@types/pg": "^8.6.5",
-    "netlify-cli": "^9.13.6"
+    "eslint": "^8.25.0",
+    "eslint-config-next": "^12.3.1",
+    "typescript": "4.9.3"
  }
 }
--- a/prisma/migrations/20220603165310_cascade_delete_frontpage/migration.sql
+++ b/prisma/migrations/20220603165310_cascade_delete_frontpage/migration.sql
@ -0,0 +1,5 @@
+-- DropForeignKey
+ALTER TABLE "FrontpageId" DROP CONSTRAINT "FrontpageId_id_fkey";
+
+-- AddForeignKey
+ALTER TABLE "FrontpageId" ADD CONSTRAINT "FrontpageId_id_fkey" FOREIGN KEY ("id") REFERENCES "questions"("id") ON DELETE CASCADE ON UPDATE CASCADE;
--- a/prisma/schema.prisma
+++ b/prisma/schema.prisma
@ -90,6 +90,6 @@ model History {
 }

 model FrontpageId {
-  question Question @relation(fields: [id], references: [id])
+  question Question @relation(fields: [id], references: [id], onDelete: Cascade)
  id       String   @unique
 }
--- a/scripts/README.md
+++ b/scripts/README.md
--- a/scripts/download-all.mjs
+++ b/scripts/download-all.mjs
--- a/scripts/download-first-seen-today.mjs
+++ b/scripts/download-first-seen-today.mjs
--- a/scripts/iframe-snippet.md
+++ b/scripts/iframe-snippet.md
--- a/src/Global.d.ts
+++ b/src/Global.d.ts
@ -0,0 +1,8 @@
+// Workaround related to: https://github.com/vercel/next.js/issues/29788
+// https://github.com/vercel/next.js/issues/29788#issuecomment-1000595524
+declare type StaticImageData = {
+  src: string;
+  height: number;
+  width: number;
+  placeholder?: string;
+};
--- a/src/pages/_middleware.ts
+++ b/src/pages/_middleware.ts
--- a/src/backend/manual/pullSuperforecastsManually.sh
+++ b/src/backend/manual/pullSuperforecastsManually.sh
@ -1,4 +1,5 @@
 #!/bin/bash

 cd /home/loki/Documents/core/software/fresh/js/metaforecast/metaforecast-monorepo
-/home/loki/.nvm/versions/node/v17.5.0/bin/npm run cli 3 > ../last-superforecast-fetch.txt
+
+/home/loki/.nvm/versions/node/v16.15.0/lib/node_modules/npm/bin/npm-cli.js run cli goodjudgment  > ../last-superforecast-fetch.txt
--- a/src/backend/platforms/goodjudgment.ts
+++ b/src/backend/platforms/goodjudgment.ts
@ -1,10 +1,11 @@
 /* Imports */
 import axios from "axios";
-import { Tabletojson } from "tabletojson";
+import {Tabletojson} from "tabletojson";

-import { average } from "../../utils";
-import { hash } from "../utils/hash";
-import { FetchedQuestion, Platform } from "./";
+import {average} from "../../utils";
+import {hash} from "../utils/hash";
+import {FetchedQuestion, Platform} from "./";
+import {FullQuestionOption} from "../../common/types";

 /* Definitions */
 const platformName = "goodjudgment";
@ -30,32 +31,30 @@ export const goodjudgment: Platform = {
    // hard-coded backup proxy
 		*/
    // proxy = {
-    //   ip: process.env.BACKUP_PROXY_IP,
-    //   port: process.env.BACKUP_PROXY_PORT,
+    // ip: process.env.BACKUP_PROXY_IP,
+    // port: process.env.BACKUP_PROXY_PORT,
    // };
    // // }
    // let agent = tunnel.httpsOverHttp({
-    //   proxy: {
+    // proxy: {
    //     host: proxy.ip,
    //     port: proxy.port,
-    //   },
+    // },
    // });

-    const content = await axios
-      .request({
-        url: "https://goodjudgment.io/superforecasts/",
-        method: "get",
-        headers: {
-          "User-Agent": "Chrome",
-        },
-        // agent,
-        // port: 80,
-      })
-      .then((query) => query.data);
+    const content = await axios.request({
+      url: "https://goodjudgment.io/superforecasts/",
+      method: "get",
+      headers: {
+        "User-Agent": "Chrome"
+      },
+      // agent,
+      // port: 80,
+    }).then((query) => query.data);

    // Processing
    let results: FetchedQuestion[] = [];
-    let jsonTable = Tabletojson.convert(content, { stripHtmlFromCells: false });
+    let jsonTable = Tabletojson.convert(content, {stripHtmlFromCells: false});
    jsonTable.shift(); // deletes first element
    jsonTable.pop(); // deletes last element

@ -63,38 +62,21 @@ export const goodjudgment: Platform = {
      let title = table[0]["0"].split("\t\t\t").splice(3)[0];
      if (title != undefined) {
        title = title.replaceAll("</a>", "");
-        const id = `${platformName}-${hash(title)}`;
-        const description = table
-          .filter((row: any) => row["0"].includes("BACKGROUND:"))
-          .map((row: any) => row["0"])
-          .map((text: any) =>
-            text
-              .split("BACKGROUND:")[1]
-              .split("Examples of Superforecaster")[0]
-              .split("AT A GLANCE")[0]
-              .replaceAll("\n\n", "\n")
-              .split("\n")
-              .slice(3)
-              .join(" ")
-              .replaceAll("      ", "")
-              .replaceAll("<br> ", "")
-          )[0];
-        const options = table
-          .filter((row: any) => "4" in row)
-          .map((row: any) => ({
-            name: row["2"]
-              .split('<span class="qTitle">')[1]
-              .replace("</span>", ""),
-            probability: Number(row["3"].split("%")[0]) / 100,
-            type: "PROBABILITY",
-          }));
-        let analysis = table.filter((row: any) =>
-          row[0] ? row[0].toLowerCase().includes("commentary") : false
-        );
+        const id = `${platformName}-${
+          hash(title)
+        }`;
+        const description = table.filter((row : any) => row["0"].includes("BACKGROUND:")).map((row : any) => row["0"]).map((text : any) => text.split("BACKGROUND:")[1].split("Examples of Superforecaster")[0].split("AT A GLANCE")[0].replaceAll("\n\n", "\n").split("\n").slice(3).join(" ").replaceAll("      ", "").replaceAll("<br> ", ""))[0];
+        const options = table.filter((row : any) => "4" in row).map((row : any) => ({
+          name: row["2"].split('<span class="qTitle">')[1].replace("</span>", ""),
+          probability: Number(row["3"].split("%")[0]) / 100,
+          type: "PROBABILITY"
+        }));
+        let analysis = table.filter((row : any) => row[0] ? row[0].toLowerCase().includes("commentary") : false);
        // "Examples of Superforecaster Commentary" / Analysis
        // The following is necessary twice, because we want to check if there is an empty list, and then get the first element of the first element of the list.
        analysis = analysis ? analysis[0] : "";
-        analysis = analysis ? analysis[0] : ""; // not a duplicate
+        analysis = analysis ? analysis[0] : "";
+        // not a duplicate
        // console.log(analysis)
        let standardObj: FetchedQuestion = {
          id,
@ -104,16 +86,14 @@ export const goodjudgment: Platform = {
          options,
          qualityindicators: {},
          extra: {
-            superforecastercommentary: analysis || "",
-          },
+            superforecastercommentary: analysis || ""
+          }
        };
        results.push(standardObj);
      }
    }

-    console.log(
-      "Failing is not unexpected; see utils/pullSuperforecastsManually.sh/js"
-    );
+    console.log("Failing is not unexpected; see utils/pullSuperforecastsManually.sh/js");

    return results;
  },
@ -121,8 +101,8 @@ export const goodjudgment: Platform = {
    let nuno = () => 4;
    let eli = () => 4;
    let misha = () => 3.5;
-    let starsDecimal = average([nuno()]); //, eli(), misha()])
+    let starsDecimal = average([nuno()]); // , eli(), misha()])
    let starsInteger = Math.round(starsDecimal);
    return starsInteger;
-  },
+  }
 };
--- a/src/backend/platforms/goodjudgmentopen.ts
+++ b/src/backend/platforms/goodjudgmentopen.ts
@ -1,12 +1,13 @@
 /* Imports */
 import axios from "axios";
-import { Tabletojson } from "tabletojson";
+import {Tabletojson} from "tabletojson";

-import { average } from "../../utils";
-import { applyIfSecretExists } from "../utils/getSecrets";
-import { sleep } from "../utils/sleep";
+import {average} from "../../utils";
+import {applyIfSecretExists} from "../utils/getSecrets";
+import {sleep} from "../utils/sleep";
 import toMarkdown from "../utils/toMarkdown";
-import { FetchedQuestion, Platform } from "./";
+import {FetchedQuestion, Platform} from "./";
+import {FullQuestionOption} from "../../common/types";

 /* Definitions */
 const platformName = "goodjudgmentopen";
@ -17,125 +18,98 @@ const annoyingPromptUrls = [
  "https://www.gjopen.com/questions/1779-are-there-any-forecasting-tips-tricks-and-experiences-you-would-like-to-share-and-or-discuss-with-your-fellow-forecasters",
  "https://www.gjopen.com/questions/2246-are-there-any-forecasting-tips-tricks-and-experiences-you-would-like-to-share-and-or-discuss-with-your-fellow-forecasters-2022-thread",
  "https://www.gjopen.com/questions/2237-what-forecasting-questions-should-we-ask-what-questions-would-you-like-to-forecast-on-gjopen",
+  "https://www.gjopen.com/questions/2437-what-forecasting-questions-should-we-ask-what-questions-would-you-like-to-forecast-on-gjopen"
 ];
 const DEBUG_MODE: "on" | "off" = "off"; // "on"
 const id = () => 0;

 /* Support functions */

-async function fetchPage(page: number, cookie: string) {
+function cleanDescription(text : string) {
+  let md = toMarkdown(text);
+  let result = md.replaceAll("---", "-").replaceAll("  ", " ");
+  return result;
+}
+
+async function fetchPage(page : number, cookie : string) {
  const response: string = await axios({
    url: htmlEndPoint + page,
    method: "GET",
    headers: {
-      Cookie: cookie,
-    },
+      Cookie: cookie
+    }
  }).then((res) => res.data);
-  //console.log(response)
+  // console.log(response)
  return response;
 }

-async function fetchStats(questionUrl: string, cookie: string) {
+async function fetchStats(questionUrl : string, cookie : string) {
  let response: string = await axios({
    url: questionUrl + "/stats",
    method: "GET",
    headers: {
+      "Content-Type": "text/html",
      Cookie: cookie,
-      Referer: questionUrl,
-    },
+      Referer: questionUrl
+    }
  }).then((res) => res.data);
-  //console.log(response)

-  // Is binary?
-  let isbinary = response.includes("binary?&quot;:true");
-
-  let options: FetchedQuestion["options"] = [];
-  if (isbinary) {
-    // Crowd percentage
-    let htmlElements = response.split("\n");
-    let h3Element = htmlElements.filter((str) => str.includes("<h3>"))[0];
-    // console.log(h3Element)
-    let crowdpercentage = h3Element.split(">")[1].split("<")[0];
-    let probability = Number(crowdpercentage.replace("%", "")) / 100;
-    options.push(
-      {
-        name: "Yes",
-        probability: probability,
-        type: "PROBABILITY",
-      },
-      {
-        name: "No",
-        probability: +(1 - probability).toFixed(2), // avoids floating point shenanigans
-        type: "PROBABILITY",
-      }
-    );
-  } else {
-    let optionsHtmlElement = "<table" + response.split("tbody")[1] + "table>";
-    let tablesAsJson = Tabletojson.convert(optionsHtmlElement);
-    let firstTable = tablesAsJson[0];
-    options = firstTable.map((element: any) => ({
-      name: element["0"],
-      probability: Number(element["1"].replace("%", "")) / 100,
-      type: "PROBABILITY",
-    }));
-    //console.log(optionsHtmlElement)
-    //console.log(options)
+  if (response.includes("Sign up or sign in to forecast")) {
+    throw Error("Not logged in");
  }
+  // Init
+  let options: FullQuestionOption[] = [];

-  // Description
-  let descriptionraw = response.split(
-    `<div id="question-background" class="collapse smb">`
-  )[1];
-  let descriptionprocessed1 = descriptionraw.split(`</div>`)[0];
-  let descriptionprocessed2 = toMarkdown(descriptionprocessed1);
-  let descriptionprocessed3 = descriptionprocessed2
-    .split("\n")
-    .filter((string) => !string.includes("Confused? Check our"))
-    .join("\n");
-  let description = descriptionprocessed3;
-
-  // Number of forecasts
-  let numforecasts = response
-    .split("prediction_sets_count&quot;:")[1]
-    .split(",")[0];
-  //console.log(numforecasts)
-
-  // Number of predictors
-  let numforecasters = response
-    .split("predictors_count&quot;:")[1]
-    .split(",")[0];
-  //console.log(numpredictors)
-
+  // Parse the embedded json
+  let htmlElements = response.split("\n");
+  let jsonLines = htmlElements.filter((element) => element.includes("data-react-props"));
+  let embeddedJsons = jsonLines.map((jsonLine, i) => {
+    let innerJSONasHTML = jsonLine.split('data-react-props="')[1].split('"')[0];
+    let json = JSON.parse(innerJSONasHTML.replaceAll("&quot;", '"'));
+    return json;
+  });
+  let firstEmbeddedJson = embeddedJsons[0];
+  let title = firstEmbeddedJson.question.name;
+  let description = cleanDescription(firstEmbeddedJson.question.description);
+  let comments_count = firstEmbeddedJson.question.comments_count;
+  let numforecasters = firstEmbeddedJson.question.predictors_count;
+  let numforecasts = firstEmbeddedJson.question.prediction_sets_count;
+  let questionType = firstEmbeddedJson.question.type;
+  if (questionType.includes("Binary") || questionType.includes("NonExclusiveOpinionPoolQuestion") || questionType.includes("Forecast::Question") || ! questionType.includes("Forecast::MultiTimePeriodQuestion")) {
+    options = firstEmbeddedJson.question.answers.map((answer : any) => ({name: answer.name, probability: answer.normalized_probability, type: "PROBABILITY"}));
+    if (options.length == 1 && options[0].name == "Yes") {
+      let probabilityNo = options[0].probability > 1 ? 1 - options[0].probability / 100 : 1 - options[0].probability;
+      options.push({name: "No", probability: probabilityNo, type: "PROBABILITY"});
+    }
+  }
  let result = {
-    description,
-    options,
+    description: description,
+    options: options,
    qualityindicators: {
      numforecasts: Number(numforecasts),
      numforecasters: Number(numforecasters),
-    },
-    // this mismatches the code below, and needs to be fixed, but I'm doing typescript conversion and don't want to touch any logic for now
-  } as any;
-
+      comments_count: Number(comments_count)
+    }
+  };
+  // console.log(JSON.stringify(result, null, 4));
  return result;
 }

-function isSignedIn(html: string) {
-  let isSignedInBool = !(
-    html.includes("You need to sign in or sign up before continuing") ||
-    html.includes("Sign up")
-  );
+function isSignedIn(html : string) {
+  let isSignedInBool = !(html.includes("You need to sign in or sign up before continuing") || html.includes("Sign up"));
  // console.log(html)
-  if (!isSignedInBool) {
+  if (! isSignedInBool) {
    console.log("Error: Not signed in.");
  }
-  console.log(`is signed in? ${isSignedInBool ? "yes" : "no"}`);
+  console.log(`is signed in? ${
+    isSignedInBool ? "yes" : "no"
+  }`);
  return isSignedInBool;
 }

-function reachedEnd(html: string) {
+function reachedEnd(html : string) {
  let reachedEndBool = html.includes("No questions match your filter");
-  if (reachedEndBool) {
-    //console.log(html)
+  if (reachedEndBool) { // console.log(html)
  }
  console.log(`Reached end? ${reachedEndBool}`);
  return reachedEndBool;
@ -143,14 +117,15 @@ function reachedEnd(html: string) {

 /* Body */

-async function goodjudgmentopen_inner(cookie: string) {
+async function goodjudgmentopen_inner(cookie : string) {
  let i = 1;
  let response = await fetchPage(i, cookie);

  let results = [];
  let init = Date.now();
  // console.log("Downloading... This might take a couple of minutes. Results will be shown.")
-  while (!reachedEnd(response) && isSignedIn(response)) {
+  console.log("Page #1")
+  while (! reachedEnd(response) && isSignedIn(response)) {
    let htmlLines = response.split("\n");
    DEBUG_MODE == "on" ? htmlLines.forEach((line) => console.log(line)) : id();
    let h5elements = htmlLines.filter((str) => str.includes("<h5> <a href="));
@ -159,20 +134,19 @@ async function goodjudgmentopen_inner(cookie: string) {
    for (let h5element of h5elements) {
      let h5elementSplit = h5element.split('"><span>');
      let url = h5elementSplit[0].split('<a href="')[1];
-      if (!annoyingPromptUrls.includes(url)) {
+      if (! annoyingPromptUrls.includes(url)) {
        let title = h5elementSplit[1].replace("</span></a></h5>", "");
        await sleep(1000 + Math.random() * 1000); // don't be as noticeable
        try {
          let moreinfo = await fetchStats(url, cookie);
-          if (moreinfo.isbinary) {
-            if (!moreinfo.crowdpercentage) {
-              // then request again.
+          /*if (moreinfo.isbinary) {
+            if (! moreinfo.crowdpercentage) { // then request again.
              moreinfo = await fetchStats(url, cookie);
            }
-          }
+          }*/
          let questionNumRegex = new RegExp("questions/([0-9]+)");
          const questionNumMatch = url.match(questionNumRegex);
-          if (!questionNumMatch) {
+          if (! questionNumMatch) {
            throw new Error(`Couldn't find question num in ${url}`);
          }
          let questionNum = questionNumMatch[1];
@ -182,19 +156,19 @@ async function goodjudgmentopen_inner(cookie: string) {
            title: title,
            url: url,
            platform: platformName,
-            ...moreinfo,
+            ... moreinfo
          };
          if (j % 30 == 0 || DEBUG_MODE == "on") {
            console.log(`Page #${i}`);
            console.log(question);
+          } else {
+            console.log(question.title)
          }
          // console.log(question)
          results.push(question);
        } catch (error) {
          console.log(error);
-          console.log(
-            `We encountered some error when fetching the URL: ${url}, so it won't appear on the final json`
-          );
+          console.log(`We encountered some error when fetching the URL: ${url}, so it won't appear on the final json`);
        }
      }
      j = j + 1;
@ -207,9 +181,7 @@ async function goodjudgmentopen_inner(cookie: string) {
      response = await fetchPage(i, cookie);
    } catch (error) {
      console.log(error);
-      console.log(
-        `We encountered some error when fetching page #${i}, so it won't appear on the final json`
-      );
+      console.log(`We encountered some error when fetching page #${i}, so it won't appear on the final json`);
    }
  }

@ -220,9 +192,11 @@ async function goodjudgmentopen_inner(cookie: string) {

  let end = Date.now();
  let difference = end - init;
-  console.log(
-    `Took ${difference / 1000} seconds, or ${difference / (1000 * 60)} minutes.`
-  );
+  console.log(`Took ${
+    difference / 1000
+  } seconds, or ${
+    difference / (1000 * 60)
+  } minutes.`);

  return results;
 }
@ -234,23 +208,18 @@ export const goodjudgmentopen: Platform = {
  version: "v1",
  async fetcher() {
    let cookie = process.env.GOODJUDGMENTOPENCOOKIE;
-    return (await applyIfSecretExists(cookie, goodjudgmentopen_inner)) || null;
+    return(await applyIfSecretExists(cookie, goodjudgmentopen_inner)) || null;
  },
  calculateStars(data) {
-    let minProbability = Math.min(
-      ...data.options.map((option) => option.probability || 0)
-    );
-    let maxProbability = Math.max(
-      ...data.options.map((option) => option.probability || 0)
-    );
+    let minProbability = Math.min(...data.options.map((option) => option.probability || 0));
+    let maxProbability = Math.max(...data.options.map((option) => option.probability || 0));

    let nuno = () => ((data.qualityindicators.numforecasts || 0) > 100 ? 3 : 2);
    let eli = () => 3;
-    let misha = () =>
-      minProbability > 0.1 || maxProbability < 0.9 ? 3.1 : 2.5;
+    let misha = () => minProbability > 0.1 || maxProbability < 0.9 ? 3.1 : 2.5;

    let starsDecimal = average([nuno(), eli(), misha()]);
    let starsInteger = Math.round(starsDecimal);
    return starsInteger;
-  },
+  }
 };
--- a/src/backend/platforms/insight.ts
+++ b/src/backend/platforms/insight.ts
@ -0,0 +1,339 @@
+/* Imports */
+import {or} from "ajv/dist/compile/codegen";
+import axios from "axios";
+
+import {FetchedQuestion, Platform} from ".";
+import {QuestionOption} from "../../common/types";
+import toMarkdown from "../utils/toMarkdown";
+import { average } from "../../utils";
+
+/* Definitions */
+const platformName = "insight";
+const marketsEnpoint = "https://insightprediction.com/api/markets?orderBy=is_resolved&sortedBy=asc";
+const getMarketEndpoint = (id : number) => `https://insightprediction.com/api/markets/${id}`;
+const SPORTS_CATEGORIES = [
+  'World Cup',
+  'MLB',
+  'Futures',
+  'Sports',
+  'EPL',
+  'Golf',
+  'NHL',
+  'College Football'
+]
+
+/* Support functions */
+
+// Stubs
+const excludeMarketFromTitle = (title : any) => {
+  if (!!title) {
+    return title.includes(" vs ") || title.includes(" Over: ") || title.includes("NFL") || title.includes("Will there be a first time winner") || title.includes("Premier League")
+  } else {
+    return true
+  }
+}
+
+const hasActiveYesNoOrderBook = (orderbook : any) => {
+  if (!!orderbook) {
+    let yes = !!orderbook.yes && !!orderbook.yes.buy && Array.isArray(orderbook.yes.buy) && orderbook.yes.buy.length != 0 && !!orderbook.yes.buy[0].price && !!orderbook.yes.sell && Array.isArray(orderbook.yes.sell) && orderbook.yes.sell.length != 0 && !!orderbook.yes.sell[0].price
+    let no = !!orderbook.no && !!orderbook.no.buy && Array.isArray(orderbook.no.buy) && orderbook.no.buy.length != 0 && !!orderbook.no.buy[0].price && !!orderbook.no.sell && Array.isArray(orderbook.no.sell) && orderbook.no.sell.length != 0 && !!orderbook.no.sell[0].price
+    return yes && no
+  } else {
+    return false
+  }
+}
+
+const isBinaryQuestion = (data : any) => Array.isArray(data) && data.length == 1
+
+const geomMean = (a : number, b : number) => Math.sqrt(a * b)
+
+const processRelativeUrls = (a : string) => a.replaceAll("] (/", "](http://insightprediction.com/").replaceAll("](/", "](http://insightprediction.com/")
+
+const processDescriptionText = (text : any) => {
+  if (typeof text === 'string') {
+    return processRelativeUrls(toMarkdown(text))
+  } else {
+    return ""
+  }
+}
+
+const getOrderbookPrize = (orderbook : any) => {
+  let yes_min_cents = orderbook.yes.buy[0].price
+  let yes_max_cents = orderbook.yes.sell[0].price
+  let yes_min = Number(yes_min_cents.slice(0, -1))
+  let yes_max = Number(yes_max_cents.slice(0, -1))
+  let yes_price_orderbook = geomMean(yes_min, yes_max)
+  return yes_price_orderbook
+}
+
+const getAnswerProbability = (answer : any) => {
+  let orderbook = answer.orderbook
+  let latest_yes_price = answer.latest_yes_price
+  if (!! orderbook && hasActiveYesNoOrderBook(orderbook)) {
+    let yes_price_orderbook = getOrderbookPrize(orderbook)
+    let yes_probability = (latest_yes_price ? geomMean(latest_yes_price, yes_price_orderbook) : yes_price_orderbook) / 100
+    return yes_probability
+  } else if (!! latest_yes_price) {
+    return latest_yes_price / 100
+  } else {
+    return -1
+  }
+}
+
+// Fetching
+async function fetchPage(bearer: string, pageNum: number) {
+  let pageUrl = `${marketsEnpoint}&page=${pageNum}`
+  const response = await axios({
+    url: pageUrl, // &orderBy=is_resolved&sortedBy=desc`,
+    method: "GET",
+    headers: {
+      "Content-Type": "application/json",
+      Accept: "application/json",
+      Authorization: `Bearer ${bearer}`
+    }
+  }).then((res) => res.data);
+  // console.log(response);
+  return response;
+}
+
+async function fetchMarket(bearer: string, marketId: number) {
+  const response = await axios({
+    url: getMarketEndpoint(marketId),
+    method: "GET",
+    headers: {
+      "Content-Type": "application/json",
+      Accept: "application/json",
+      Authorization: `Bearer ${bearer}`
+    }
+  }).then((res) => res.data);
+  // console.log(response)
+  return response;
+
+}
+
+const processMarket = (market : any) => {
+  let options: FetchedQuestion["options"] = []
+
+  if (!!market && !!market.answer && !!market.answer.data) {
+    let data = market.answer.data
+    if (isBinaryQuestion(data)) { // Binary questions
+      let answer = data[0]
+      let probability = getAnswerProbability(answer)
+      if (probability != -1) {
+        options = [
+          {
+            name: "Yes",
+            probability: probability,
+            type: "PROBABILITY"
+          }, {
+            name: "No",
+            probability: 1 - probability,
+            type: "PROBABILITY"
+          },
+        ];
+      } 
+    } else { // non binary question
+      for (let answer of data) {
+        let probability = getAnswerProbability(answer)
+        if (probability != -1) {
+          let newOption: QuestionOption = ({
+            name: String(answer.title),
+            probability: probability,
+            type: "PROBABILITY"
+          });
+          options.push(newOption)
+        }
+      }
+    }
+    if (!! options && Array.isArray(options) && options.length > 0) {
+      const id = `${platformName}-${
+        market.id
+      }`
+      const result: FetchedQuestion = {
+        id: id,
+        title: market.title,
+        url: market.url,
+        description: processDescriptionText(market.rules),
+        options,
+        qualityindicators: market.coin_id == "USD" ? (
+          {volume: market.volume}
+        ) : ({})
+      };
+      return result;
+    }
+  } 
+  return null
+}
+
+async function fetchAllMarkets(bearer: string) {
+  let pageNum = 1
+  let markets = []
+  let categories = []
+  let isEnd = false
+  while (! isEnd) {
+    if(pageNum % 20 == 0){
+      console.log(`Fetching page #${pageNum}`) // : ${pageUrl}
+    }
+    let page = await fetchPage(bearer, pageNum)
+    // console.log(JSON.stringify(page, null, 2))
+    let data = page.data
+    if (!! data && Array.isArray(data) && data.length > 0) {
+      let lastMarket = data[data.length - 1]
+      let isLastMarketResolved = lastMarket.is_resolved
+      if (isLastMarketResolved == true) {
+        isEnd = true
+      }
+      let newMarkets = data.filter(market => !market.is_resolved && !market.is_expired && ! excludeMarketFromTitle(market.title))
+      for (let initMarketData of newMarkets) {
+        let fullMarketDataResponse = await fetchMarket(bearer, initMarketData.id)
+        let fullMarketData = fullMarketDataResponse.data
+        let processedMarketData = processMarket(fullMarketData)
+
+        if (processedMarketData != null && ! SPORTS_CATEGORIES.includes(fullMarketData.category)) {
+          console.log(`- Adding: ${
+            fullMarketData.title
+          }`)
+          console.group()
+          console.log(fullMarketData)
+          console.log(JSON.stringify(processedMarketData, null, 2))
+          console.groupEnd()
+
+          markets.push(processedMarketData)
+        }
+
+        let category = fullMarketData.category
+        categories.push(category)
+
+      }
+    } else {
+      isEnd = true
+    } pageNum = pageNum + 1
+  }
+  console.log(markets)
+  console.log(categories)
+  return markets
+}
+/*
+async function fetchQuestionStats(bearer : string, marketId : number) {
+  const response = await axios({
+    url: getMarketEndpoint(marketId),
+    method: "GET",
+    headers: {
+      "Content-Type": "application/json",
+      Accept: "application/json",
+      Authorization: `Bearer ${bearer}`
+    }
+  }).then((res) => res.data);
+  // console.log(response)
+  return response;
+}
+
+
+
+async function fetchData(bearer : string) {
+  let pageNum = 1;
+  let reachedEnd = false;
+  let results = [];
+  while (! reachedEnd) {
+    let newPage = await fetchPage(bearer, pageNum);
+    let newPageData = newPage.data;
+    let marketsFromPage = []
+    for (let market of newPageData) {
+      let response = await fetchQuestionStats(bearer, market.id);
+      let marketData = response.data
+      let marketAnswer = marketData.answer.data
+      delete marketData.answer
+      // These are the options and their prices.
+      let marketOptions = marketAnswer.map(answer => {
+        return({name: answer.title, probability: answer.latest_yes_price, type: "PROBABILITY"})
+      })
+      marketsFromPage.push({
+        ... marketData,
+        options: marketOptions
+      });
+    }
+
+    let finalObject = marketsFromPage
+
+    console.log(`Page = #${pageNum}`);
+    // console.log(newPageData)
+    console.dir(finalObject, {depth: null});
+    results.push(... finalObject);
+
+    let newPagination = newPage.meta.pagination;
+    if (newPagination.total_pages == pageNum) {
+      reachedEnd = true;
+    } else {
+      pageNum = pageNum + 1;
+    }
+  }
+  return results
+}
+
+async function processPredictions(predictions : any[]) {
+  let results = await predictions.map((prediction) => {
+    const id = `${platformName}-${
+      prediction.id
+    }`;
+    const probability = prediction.probability;
+    const options: FetchedQuestion["options"] = [
+      {
+        name: "Yes",
+        probability: probability,
+        type: "PROBABILITY"
+      }, {
+        name: "No",
+        probability: 1 - probability,
+        type: "PROBABILITY"
+      },
+    ];
+    const result: FetchedQuestion = {
+      id,
+      title: prediction.title,
+      url: "https://example.com",
+      description: prediction.description,
+      options,
+      qualityindicators: {
+        // other: prediction.otherx,
+        // indicators: prediction.indicatorx,
+      }
+    };
+    return result;
+  });
+  return results; // resultsProcessed
+}
+*/
+/* Body */
+export const insight: Platform = {
+  name: platformName,
+  label: "Insight Prediction",
+  color: "#ff0000",
+  version: "v1",
+  async fetcher() {
+    let bearer = process.env.INSIGHT_BEARER;
+    if (!! bearer) {
+      let data = await fetchAllMarkets(bearer);
+      return data
+    } else {
+      throw Error("No INSIGHT_BEARER available in environment")
+    }
+    // let results: FetchedQuestion[] = []; // await processPredictions(data); // somehow needed
+    // return results;
+  },
+  calculateStars(data) {
+    let nuno = () => {
+      if((data.qualityindicators.volume || 0) > 10000){
+        return 4
+      } else if((data.qualityindicators.volume || 0) > 1000){
+        return 3
+      } else{
+        return 2
+      }
+    }
+    let eli = () => null;
+    let misha = () => null;
+    let starsDecimal = average([nuno()]); //, eli(data), misha(data)])
+    let starsInteger = Math.round(starsDecimal);
+    return starsInteger;
+  }
+};
--- a/src/backend/platforms/manifold.ts
+++ b/src/backend/platforms/manifold.ts
@ -6,12 +6,12 @@ import { FetchedQuestion, Platform } from "./";

 /* Definitions */
 const platformName = "manifold";
-const endpoint = "https://manifold.markets/api/v0/markets";
+const ENDPOINT = "https://manifold.markets/api/v0/markets";
 // See https://manifoldmarkets.notion.site/Manifold-Markets-API-5e7d0aef4dcf452bb04b319e178fabc5

 /* Support functions */

-async function fetchData() {
+async function fetchPage(endpoint: string) {
  let response = await axios({
    url: endpoint,
    method: "GET",
@ -23,6 +23,31 @@ async function fetchData() {
  return response;
 }

+async function fetchAllData(){
+  let endpoint = ENDPOINT
+  let end = false
+  let allData = []
+  let counter = 1
+  while(!end){
+    console.log(`Query #${counter}: ${endpoint}`)
+    let newData = await fetchPage(endpoint)
+    if(Array.isArray(newData)){
+      allData.push(...newData)
+      let hasReachedEnd = (newData.length == 0) || (newData[newData.length -1] == undefined) || (newData[newData.length -1].id == undefined)
+      if(!hasReachedEnd){
+        let lastId = newData[newData.length -1].id
+        endpoint = `${ENDPOINT}?before=${lastId}`
+      }else{
+        end = true
+      }
+    }else{
+      end = true
+    }
+    counter = counter +1
+  }
+  return allData
+}
+
 function showStatistics(results: FetchedQuestion[]) {
  console.log(`Num unresolved markets: ${results.length}`);
  let sum = (arr: number[]) => arr.reduce((tally, a) => tally + a, 0);
@ -63,12 +88,12 @@ function processPredictions(predictions: any[]): FetchedQuestion[] {
      id: id,
      title: prediction.question,
      url: prediction.url,
-      description: prediction.description,
+      description: prediction.description || "",
      options,
      qualityindicators: {
        createdTime: prediction.createdTime,
-        volume7Days: prediction.volume7Days,
-        volume24Hours: prediction.volume24Hours,
+        // volume7Days: prediction.volume7Days, // deprecated.
+        volume24Hours: prediction.volume24Hours, 
        pool: prediction.pool, // normally liquidity, but I don't actually want to show it.
      },
      extra: {
@ -90,16 +115,16 @@ export const manifold: Platform = {
  color: "#793466",
  version: "v1",
  async fetcher() {
-    let data = await fetchData();
+    let data = await fetchAllData();
    let results = processPredictions(data); // somehow needed
    showStatistics(results);
    return results;
  },
  calculateStars(data) {
    let nuno = () =>
-      (data.qualityindicators.volume7Days || 0) > 250 ||
+      (data.qualityindicators.volume24Hours || 0) > 100 ||
      ((data.qualityindicators.pool || 0) > 500 &&
-        (data.qualityindicators.volume7Days || 0) > 100)
+        (data.qualityindicators.volume24Hours || 0) > 50)
        ? 2
        : 1;
    let eli = () => null;
--- a/src/backend/platforms/metaculus.ts
+++ b/src/backend/platforms/metaculus.ts
@ -1,205 +0,0 @@
-/* Imports */
-import axios from "axios";
-
-import { average } from "../../utils";
-import { sleep } from "../utils/sleep";
-import toMarkdown from "../utils/toMarkdown";
-import { FetchedQuestion, Platform } from "./";
-
-/* Definitions */
-const platformName = "metaculus";
-let now = new Date().toISOString();
-let DEBUG_MODE = "off";
-let SLEEP_TIME = 5000;
-
-/* Support functions */
-async function fetchMetaculusQuestions(next: string) {
-  // Numbers about a given address: how many, how much, at what price, etc.
-  let response;
-  let data;
-  try {
-    response = await axios({
-      url: next,
-      method: "GET",
-      headers: { "Content-Type": "application/json" },
-    });
-    data = response.data;
-  } catch (error) {
-    console.log(`Error in async function fetchMetaculusQuestions(next)`);
-    console.log(error);
-    if (axios.isAxiosError(error)) {
-      if (error.response?.headers["retry-after"]) {
-        const timeout = error.response.headers["retry-after"];
-        console.log(`Timeout: ${timeout}`);
-        await sleep(Number(timeout) * 1000 + SLEEP_TIME);
-      } else {
-        await sleep(SLEEP_TIME);
-      }
-    }
-  } finally {
-    try {
-      response = await axios({
-        url: next,
-        method: "GET",
-        headers: { "Content-Type": "application/json" },
-      });
-      data = response.data;
-    } catch (error) {
-      console.log(error);
-      return { results: [] };
-    }
-  }
-  // console.log(response)
-  return data;
-}
-
-async function fetchMetaculusQuestionDescription(slug: string) {
-  try {
-    let response = await axios({
-      method: "get",
-      url: "https://www.metaculus.com" + slug,
-    }).then((response) => response.data);
-    return response;
-  } catch (error) {
-    console.log(`Error in: fetchMetaculusQuestionDescription`);
-    console.log(
-      `We encountered some error when attempting to fetch a metaculus page. Trying again`
-    );
-    if (
-      axios.isAxiosError(error) &&
-      typeof error.response != "undefined" &&
-      typeof error.response.headers != "undefined" &&
-      typeof error.response.headers["retry-after"] != "undefined"
-    ) {
-      const timeout = error.response.headers["retry-after"];
-      console.log(`Timeout: ${timeout}`);
-      await sleep(Number(timeout) * 1000 + SLEEP_TIME);
-    } else {
-      await sleep(SLEEP_TIME);
-    }
-    try {
-      let response = await axios({
-        method: "get",
-        url: "https://www.metaculus.com" + slug,
-      }).then((response) => response.data);
-      // console.log(response)
-      return response;
-    } catch (error) {
-      console.log(
-        `We encountered some error when attempting to fetch a metaculus page.`
-      );
-      console.log("Error", error);
-      throw "Giving up";
-    }
-  }
-}
-
-export const metaculus: Platform = {
-  name: platformName,
-  label: "Metaculus",
-  color: "#006669",
-  version: "v1",
-  async fetcher() {
-    // let metaculusQuestionsInit = await fetchMetaculusQuestions(1)
-    // let numQueries = Math.round(Number(metaculusQuestionsInit.count) / 20)
-    // console.log(`Downloading... This might take a while. Total number of queries: ${numQueries}`)
-    // for (let i = 4; i <= numQueries; i++) { // change numQueries to 10 if one want to just test }
-    let all_questions = [];
-    let next = "https://www.metaculus.com/api2/questions/";
-    let i = 1;
-    while (next) {
-      if (i % 20 == 0) {
-        console.log("Sleeping for 500ms");
-        await sleep(SLEEP_TIME);
-      }
-      console.log(`\nQuery #${i}`);
-      let metaculusQuestions = await fetchMetaculusQuestions(next);
-      let results = metaculusQuestions.results;
-      let j = false;
-      for (let result of results) {
-        if (result.publish_time < now && now < result.resolve_time) {
-          await sleep(SLEEP_TIME / 2);
-          let questionPage = await fetchMetaculusQuestionDescription(
-            result.page_url
-          );
-          if (!questionPage.includes("A public prediction by")) {
-            // console.log(questionPage)
-            let descriptionraw = questionPage.split(
-              `<div class="content" ng-bind-html-compile="qctrl.question.description_html">`
-            )[1]; //.split(`<div class="question__content">`)[1]
-            let descriptionprocessed1 = descriptionraw.split("</div>")[0];
-            let descriptionprocessed2 = toMarkdown(descriptionprocessed1);
-            let description = descriptionprocessed2;
-
-            let isbinary = result.possibilities.type == "binary";
-            let options: FetchedQuestion["options"] = [];
-            if (isbinary) {
-              let probability = Number(result.community_prediction.full.q2);
-              options = [
-                {
-                  name: "Yes",
-                  probability: probability,
-                  type: "PROBABILITY",
-                },
-                {
-                  name: "No",
-                  probability: 1 - probability,
-                  type: "PROBABILITY",
-                },
-              ];
-            }
-            let id = `${platformName}-${result.id}`;
-            let interestingInfo: FetchedQuestion = {
-              id,
-              title: result.title,
-              url: "https://www.metaculus.com" + result.page_url,
-              description,
-              options,
-              qualityindicators: {
-                numforecasts: Number(result.number_of_predictions),
-              },
-              extra: {
-                resolution_data: {
-                  publish_time: result.publish_time,
-                  resolution: result.resolution,
-                  close_time: result.close_time,
-                  resolve_time: result.resolve_time,
-                },
-              },
-              //"status": result.status,
-              //"publish_time": result.publish_time,
-              //"close_time": result.close_time,
-              //"type": result.possibilities.type, // We want binary ones here.
-              //"last_activity_time": result.last_activity_time,
-            };
-            if (Number(result.number_of_predictions) >= 10) {
-              console.log(`- ${interestingInfo.title}`);
-              all_questions.push(interestingInfo);
-              if ((!j && i % 20 == 0) || DEBUG_MODE == "on") {
-                console.log(interestingInfo);
-                j = true;
-              }
-            }
-          } else {
-            console.log("- [Skipping public prediction]");
-          }
-        }
-      }
-      next = metaculusQuestions.next;
-      i = i + 1;
-    }
-
-    return all_questions;
-  },
-
-  calculateStars(data) {
-    const { numforecasts } = data.qualityindicators;
-    let nuno = () =>
-      (numforecasts || 0) > 300 ? 4 : (numforecasts || 0) > 100 ? 3 : 2;
-    let eli = () => 3;
-    let misha = () => 3;
-    let starsDecimal = average([nuno(), eli(), misha()]);
-    let starsInteger = Math.round(starsDecimal);
-    return starsInteger;
-  },
-};
--- a/src/backend/platforms/metaculus/api.ts
+++ b/src/backend/platforms/metaculus/api.ts
@ -0,0 +1,262 @@
+import Ajv, { JTDDataType, ValidateFunction } from "ajv/dist/jtd";
+import axios from "axios";
+import { sleep } from "../../utils/sleep";
+
+// Type examples:
+// - group: https://www.metaculus.com/api2/questions/9866/
+// - claim: https://www.metaculus.com/api2/questions/9668/
+// - subquestion forecast: https://www.metaculus.com/api2/questions/10069/
+// - basic forecast: https://www.metaculus.com/api2/questions/11005/
+
+const RETRY_SLEEP_TIME = 5000;
+
+const commonProps = {
+  id: {
+    type: "uint32",
+  },
+  title: {
+    type: "string",
+  },
+} as const;
+
+const predictableProps = {
+  publish_time: {
+    type: "string",
+  },
+  close_time: {
+    type: "string",
+  },
+  resolve_time: {
+    type: "string",
+  },
+  resolution: {
+    type: "float64",
+    nullable: true,
+  },
+  possibilities: {
+    properties: {
+      type: {
+        // Enum["binary", "continuous"], via https://github.com/quantified-uncertainty/metaforecast/pull/84#discussion_r878240875
+        // but metaculus might add new values in the future and we don't want the fetcher to break
+        type: "string",
+      },
+    },
+    additionalProperties: true,
+  },
+  number_of_predictions: {
+    type: "uint32",
+  },
+  community_prediction: {
+    properties: {
+      full: {
+        // q1/q2/q3 can be missing, e.g. https://www.metaculus.com/api2/questions/1633/
+        optionalProperties: {
+          q1: {
+            type: "float64",
+          },
+          q2: {
+            type: "float64",
+          },
+          q3: {
+            type: "float64",
+          },
+        },
+        additionalProperties: true,
+      },
+    },
+    nullable: true,
+    additionalProperties: true,
+  },
+} as const;
+
+const pageProps = {
+  page_url: {
+    type: "string",
+  },
+  group: {
+    type: "uint32",
+    nullable: true,
+  },
+} as const;
+
+// these are missing in /api2/questions/ requests, and building two schemas is too much pain
+const optionalPageProps = {
+  description: {
+    type: "string",
+  },
+  description_html: {
+    type: "string",
+  },
+} as const;
+
+const questionSchema = {
+  discriminator: "type",
+  mapping: {
+    forecast: {
+      properties: {
+        ...commonProps,
+        ...pageProps,
+        ...predictableProps,
+      },
+      optionalProperties: {
+        ...optionalPageProps,
+      },
+      additionalProperties: true,
+    },
+    group: {
+      properties: {
+        ...commonProps,
+        ...pageProps,
+      },
+      optionalProperties: {
+        ...optionalPageProps,
+        sub_questions: {
+          elements: {
+            properties: {
+              ...commonProps,
+              ...predictableProps,
+            },
+            additionalProperties: true,
+          },
+        },
+      },
+      additionalProperties: true,
+    },
+    // we're not interested in claims currently (but we should be?)
+    claim: {
+      properties: {
+        ...commonProps,
+        ...pageProps,
+      },
+      optionalProperties: {
+        ...optionalPageProps,
+      },
+      additionalProperties: true,
+    },
+    discussion: {
+      optionalProperties: {
+        ...optionalPageProps,
+      },
+      additionalProperties: true,
+    },
+  },
+} as const;
+
+const knownQuestionTypes = Object.keys(questionSchema.mapping);
+
+const shallowMultipleQuestionsSchema = {
+  properties: {
+    results: {
+      elements: {
+        properties: {
+          type: {
+            type: "string",
+          },
+        },
+        additionalProperties: true,
+      },
+    },
+    next: {
+      type: "string",
+      nullable: true,
+    },
+  },
+  additionalProperties: true,
+} as const;
+
+export type ApiCommon = JTDDataType<{
+  properties: typeof commonProps;
+}>;
+export type ApiPredictable = JTDDataType<{
+  properties: typeof predictableProps;
+}>;
+export type ApiQuestion = JTDDataType<typeof questionSchema>;
+
+type ApiShallowMultipleQuestions = JTDDataType<
+  typeof shallowMultipleQuestionsSchema
+>;
+
+export type ApiMultipleQuestions = {
+  results: ApiQuestion[];
+  next: ApiShallowMultipleQuestions["next"]; // Omit<ApiShallowMultipleQuestions, "results"> doesn't work correctly here
+};
+
+const validateQuestion = new Ajv().compile<ApiQuestion>(questionSchema);
+const validateShallowMultipleQuestions =
+  new Ajv().compile<ApiShallowMultipleQuestions>(
+    shallowMultipleQuestionsSchema
+  );
+
+async function fetchWithRetries<T = unknown>(url: string): Promise<T> {
+  try {
+    const response = await axios.get<T>(url);
+    return response.data;
+  } catch (error) {
+    console.log(`Error while fetching ${url}`);
+    console.log(error);
+    if (axios.isAxiosError(error)) {
+      if (error.response?.headers["retry-after"]) {
+        const timeout = error.response.headers["retry-after"];
+        console.log(`Timeout: ${timeout}`);
+        await sleep(Number(timeout) * 1000 + 1000);
+      } else {
+        await sleep(RETRY_SLEEP_TIME);
+      }
+    }
+  }
+  const response = await axios.get<T>(url);
+  return response.data;
+}
+
+const fetchAndValidate = async <T = unknown>(
+  url: string,
+  validator: ValidateFunction<T>
+): Promise<T> => {
+  // console.log(url);
+  const data = await fetchWithRetries<object>(url);
+  if (validator(data)) {
+    return data;
+  }else{
+    console.log(data)
+    throw new Error(
+      `Response validation for url ${url} failed: ` +
+        JSON.stringify(validator.errors, null, 4)
+    );
+  }
+};
+
+export async function fetchApiQuestions(
+  next: string
+): Promise<ApiMultipleQuestions> {
+  const data = await fetchAndValidate(next, validateShallowMultipleQuestions);
+
+  const isDefined = <T>(argument: T | undefined): argument is T => {
+    return argument !== undefined;
+  };
+
+  return {
+    ...data,
+    results: data.results
+      .map((result) => {
+        if (!knownQuestionTypes.includes(result.type)) {
+          console.warn(`Unknown result type ${result.type}, skipping`);
+          return undefined;
+        }
+        if (!validateQuestion(result)) {
+          throw new Error(
+            `Response validation failed: ` +
+              JSON.stringify(validateQuestion.errors)
+          );
+        }
+        return result;
+      })
+      .filter(isDefined),
+  };
+}
+
+export async function fetchSingleApiQuestion(id: number): Promise<ApiQuestion> {
+  return await fetchAndValidate(
+    `https://www.metaculus.com/api2/questions/${id}/`,
+    validateQuestion
+  );
+}
--- a/src/backend/platforms/metaculus/index.ts
+++ b/src/backend/platforms/metaculus/index.ts
@ -0,0 +1,220 @@
+import Error from "next/error";
+import {FetchedQuestion, Platform} from "..";
+import {average} from "../../../utils";
+import {sleep} from "../../utils/sleep";
+import {
+  ApiCommon,
+  ApiMultipleQuestions,
+  ApiPredictable,
+  ApiQuestion,
+  fetchApiQuestions,
+  fetchSingleApiQuestion
+} from "./api";
+
+const platformName = "metaculus";
+const now = new Date().toISOString();
+const SLEEP_TIME = 1000;
+
+async function apiQuestionToFetchedQuestions(apiQuestion: ApiQuestion): Promise<FetchedQuestion[]> {
+  // one item can expand:
+  // - to 0 questions if we don't want it;
+  // - to 1 question if it's a simple forecast
+  // - to multiple questions if it's a group (see https://github.com/quantified-uncertainty/metaforecast/pull/84 for details)
+
+  const skip = (q : ApiPredictable) : boolean => {
+    if (q.publish_time > now || now > q.resolve_time) {
+      return true;
+    }
+    if (q.number_of_predictions < 10) {
+      return true;
+    }
+    return false;
+  };
+
+  const buildFetchedQuestion = (q : ApiPredictable & ApiCommon) : Omit < FetchedQuestion,
+    "url" | "description" | "title" > => {
+      const isBinary = q.possibilities.type === "binary";
+      let options: FetchedQuestion["options"] = [];
+      if (isBinary) {
+        const probability = q.community_prediction?.full.q2;
+        if (probability !== undefined) {
+          options = [
+            {
+              name: "Yes",
+              probability: probability,
+              type: "PROBABILITY"
+            }, {
+              name: "No",
+              probability: 1 - probability,
+              type: "PROBABILITY"
+            },
+          ];
+        }
+      }
+      return {
+          id: `${platformName}-${
+          q.id
+        }`,
+        options,
+        qualityindicators: {
+          numforecasts: q.number_of_predictions
+        },
+        extra: {
+          resolution_data: {
+            publish_time: apiQuestion.publish_time,
+            resolution: apiQuestion.resolution,
+            close_time: apiQuestion.close_time,
+            resolve_time: apiQuestion.resolve_time
+          }
+        }
+      };
+    };
+
+  if (apiQuestion.type === "group") {
+    await sleep(SLEEP_TIME);
+    let apiQuestionDetailsTemp
+    try{
+      apiQuestionDetailsTemp = await fetchSingleApiQuestion(apiQuestion.id);
+    }catch(error){
+      console.log(error)
+      return []
+    }
+    const apiQuestionDetails = apiQuestionDetailsTemp
+    if (apiQuestionDetails.type !== "group") {
+      console.log("Error: expected `group` type")
+      return [] //throw new Error("Expected `group` type"); // shouldn't happen, this is mostly for typescript
+    }else{
+      try{
+        let result = (apiQuestionDetails.sub_questions || []).filter((q) => ! skip(q)).map((sq) => {
+          const tmp = buildFetchedQuestion(sq);
+          return {
+            ... tmp,
+            title: `${
+              apiQuestion.title
+            } (${
+              sq.title
+            })`,
+            description: apiQuestionDetails.description || "",
+            url: `https://www.metaculus.com${
+              apiQuestion.page_url
+            }?sub-question=${
+              sq.id
+            }`
+          };
+        });
+        return result
+      }catch(error){
+        console.log(error)
+        return []
+      }
+    }
+      
+  } else if (apiQuestion.type === "forecast") {
+    if (apiQuestion.group) {
+      return []; // sub-question, should be handled on the group level
+    }
+    if (skip(apiQuestion)) {
+      console.log(`- [Skipping]: ${
+        apiQuestion.title
+      }`)
+      /*console.log(`Close time: ${
+        apiQuestion.close_time
+      }, resolve time: ${
+        apiQuestion.resolve_time
+      }`)*/
+      return [];
+    }
+
+    await sleep(SLEEP_TIME);
+    try{
+      const apiQuestionDetails = await fetchSingleApiQuestion(apiQuestion.id);
+      const tmp = buildFetchedQuestion(apiQuestion);
+      return [{
+          ... tmp,
+          title: apiQuestion.title,
+          description: apiQuestionDetails.description || "",
+          url: "https://www.metaculus.com" + apiQuestion.page_url
+        },];
+    }catch(error){
+      console.log(error)
+      return []
+    }
+  } else {
+    if (apiQuestion.type !== "claim") { // should never happen, since `discriminator` in JTD schema causes a strict runtime check
+      console.log(`Unknown metaculus question type: ${
+        (apiQuestion as any).type
+      }, skipping`);
+    }
+    return [];
+  }
+}
+
+export const metaculus: Platform<"id" | "debug"> = {
+  name: platformName,
+  label: "Metaculus",
+  color: "#006669",
+  version: "v2",
+  fetcherArgs: [
+    "id", "debug"
+  ],
+  async fetcher(opts) {
+    let allQuestions: FetchedQuestion[] = [];
+
+    if (opts.args ?. id) {
+      try{
+        console.log("Using optional id arg.")
+        const id = Number(opts.args.id);
+        const apiQuestion = await fetchSingleApiQuestion(id);
+        const questions = await apiQuestionToFetchedQuestions(apiQuestion);
+        console.log(questions);
+        return {questions, partial: true};
+  
+      }catch(error){
+        console.log(error)
+        return {questions: [], partial: true};
+      }
+    }
+
+    let next: string | null = "https://www.metaculus.com/api2/questions/";
+    let i = 1;
+    while (next) {
+      console.log(`\nQuery #${i} - ${next}`);
+
+      await sleep(SLEEP_TIME);
+      const apiQuestions: ApiMultipleQuestions = await fetchApiQuestions(next);
+      const results = apiQuestions.results;
+      // console.log(results)
+      let j = false;
+
+      for (const result of results) {
+        const questions = await apiQuestionToFetchedQuestions(result);
+        // console.log(questions)
+        for (const question of questions) {
+          console.log(`- ${
+            question.title
+          }`);
+          if ((! j && i % 20 === 0) || opts.args ?. debug) {
+            console.log(question);
+            j = true;
+          }
+          allQuestions.push(question);
+        }
+      }
+
+      next = apiQuestions.next;
+      i += 1;
+    }
+
+    return {questions: allQuestions, partial: false};
+  },
+
+  calculateStars(data) {
+    const {numforecasts} = data.qualityindicators;
+    const nuno = () => (numforecasts || 0) > 300 ? 4 : (numforecasts || 0) > 100 ? 3 : 2;
+    const eli = () => 3;
+    const misha = () => 3;
+    const starsDecimal = average([nuno(), eli(), misha()]);
+    const starsInteger = Math.round(starsDecimal);
+    return starsInteger;
+  }
+};
--- a/src/backend/platforms/registry.ts
+++ b/src/backend/platforms/registry.ts
@ -7,6 +7,7 @@ import { goodjudgmentopen } from "./goodjudgmentopen";
 import { guesstimate } from "./guesstimate";
 import { Platform, PlatformConfig } from "./index";
 import { infer } from "./infer";
+import { insight } from "./insight";
 import { kalshi } from "./kalshi";
 import { manifold } from "./manifold";
 import { metaculus } from "./metaculus";
@ -28,6 +29,7 @@ export const getPlatforms = (): Platform<string>[] => {
    goodjudgmentopen,
    guesstimate,
    infer,
+    insight,
    kalshi,
    manifold,
    metaculus,
--- a/src/pages/_app.tsx
+++ b/src/pages/_app.tsx
@ -17,12 +17,14 @@ Router.events.on("routeChangeStart", (as, { shallow }) => {
 Router.events.on("routeChangeComplete", () => NProgress.done());
 Router.events.on("routeChangeError", () => NProgress.done());

+
 function MyApp({ Component, pageProps }: AppProps) {
  return (
    <PlausibleProvider domain="metaforecast.org">
      <Component {...pageProps} />
    </PlausibleProvider>
  );
+  // Workaround in package.json for: https://github.com/vercel/next.js/issues/36019#issuecomment-1103266481
 }

 export default withUrqlClient((ssr) => getUrqlClientOptions(ssr), {
--- a/src/pages/api/graphql.ts
+++ b/src/pages/api/graphql.ts
@ -1,13 +1,24 @@
-import { NextApiRequest, NextApiResponse } from "next";
+import {NextApiRequest, NextApiResponse} from "next";

 // apollo-server-micro is problematic since v3, see https://github.com/apollographql/apollo-server/issues/5547, so we use graphql-yoga instead
-import { createServer } from "@graphql-yoga/node";
+import {createYoga} from "graphql-yoga";
+import {useResponseCache} from '@graphql-yoga/plugin-response-cache'

-import { schema } from "../../graphql/schema";
+import {schema} from "../../graphql/schema";

-const server = createServer<{
+const server = createYoga < {
  req: NextApiRequest;
  res: NextApiResponse;
-}>({ schema });
+} > ({
+  schema,
+  graphqlEndpoint: '/api/graphql',
+  plugins: [useResponseCache(
+      { // global cache
+        session: () => null,
+        ttl: 2 * 60 * 60 * 1000,
+        // ^ 2h * 60 mins per hour, 60 seconds per min 1000 miliseconds per second
+      }
+    )]
+});

 export default server;
--- a/src/web/common/BoxedLink.tsx
+++ b/src/web/common/BoxedLink.tsx
@ -0,0 +1,23 @@
+import { FaExternalLinkAlt } from "react-icons/fa";
+
+type Props = {
+  url: string;
+  size?: "normal" | "small";
+};
+
+export const BoxedLink: React.FC<Props> = ({
+  url,
+  size = "normal",
+  children,
+}) => (
+  <a
+    className={`px-2 py-1 border-2 border-gray-400 rounded-lg text-black no-underline  hover:bg-gray-100 inline-flex flex-nowrap space-x-1 items-center text-xs md:text-lg  ${
+      size === "small" ? "text-sm" : ""
+    }`}
+    href={url}
+    target="_blank"
+  >
+    <span>{children}</span>
+    <FaExternalLinkAlt className="text-gray-400 inline " />
+  </a>
+);
--- a/src/web/common/Collapsible.tsx
+++ b/src/web/common/Collapsible.tsx
@ -0,0 +1,44 @@
+import { useState } from "react";
+import { FaCaretDown, FaCaretRight } from "react-icons/fa";
+
+export const Collapsible: React.FC<{
+  title: string;
+  children: () => React.ReactElement | null;
+}> = ({ title, children }) => {
+  const [open, setOpen] = useState(false);
+
+  const expand = (e: React.SyntheticEvent) => {
+    e.preventDefault();
+    setOpen(true);
+  };
+
+  const collapse = (e: React.SyntheticEvent) => {
+    e.preventDefault();
+    setOpen(false);
+  };
+
+  if (open) {
+    return (
+      <div>
+        <a
+          href="#"
+          className="decoration-dashed inline-flex items-center"
+          onClick={collapse}
+        >
+          {title} <FaCaretDown />
+        </a>
+        <div>{children()}</div>
+      </div>
+    );
+  } else {
+    return (
+      <a
+        href="#"
+        className="decoration-dashed inline-flex items-center"
+        onClick={expand}
+      >
+        {title} <FaCaretRight />
+      </a>
+    );
+  }
+};
--- a/src/web/common/Spinner.tsx
+++ b/src/web/common/Spinner.tsx
@ -11,7 +11,7 @@ export const Spinner: React.FC = () => (
      cy="12"
      r="10"
      stroke="currentColor"
-      stroke-width="4"
+      strokeWidth="4"
    ></circle>
    <path
      className="opacity-75"
--- a/src/web/questions/components/CaptureQuestion.tsx
+++ b/src/web/questions/components/CaptureQuestion.tsx
@ -1,6 +1,6 @@
 import domtoimage from "dom-to-image"; // https://github.com/tsayen/dom-to-image
+import { Resizable } from "re-resizable";
 import { useEffect, useRef, useState } from "react";
-
 import { Button } from "../../common/Button";
 import { CopyParagraph } from "../../common/CopyParagraph";
 import { QuestionFragment } from "../../fragments.generated";
@ -79,6 +79,15 @@ const MetaculusSource: React.FC<{
  );
 };

+const GrayContainer: React.FC<{ title: string }> = ({ title, children }) => (
+  <div className="bg-gray-100 p-2 space-y-1">
+    <div className="uppercase text-xs font-bold tracking-wide text-gray-600">
+      {title}:
+    </div>
+    <div>{children}</div>
+  </div>
+);
+
 interface Props {
  question: QuestionFragment;
 }
@ -118,35 +127,55 @@ export const CaptureQuestion: React.FC<Props> = ({ question }) => {
    await exportAsPictureAndCode();
  };

-  return (
-    <div className="grid grid-cols-1 md:grid-cols-2 gap-4 place-items-center">
-      <div ref={containerRef}>
-        <QuestionCard
-          question={question}
-          showTimeStamp={true}
-          showExpandButton={false}
-          expandFooterToFullWidth={true}
-        />
-      </div>
-      <div>
-        <Button onClick={onCaptureButtonClick}>{mainButtonText}</Button>
-      </div>
-      {imgSrc ? (
-        <>
-          <div>
+  if (imgSrc) {
+    return (
+      <div className="space-y-4">
+        <GrayContainer title="Generated image">
+          <a href={imgSrc} target="_blank">
            <img src={imgSrc} />
+          </a>
+        </GrayContainer>
+        <div>
+          <ImageSource question={question} imgSrc={imgSrc} />
+        </div>
+        {question.platform.id === "metaculus" ? (
+          <>
+            <div className="justify-self-stretch">
+              <MetaculusEmbed question={question} />
+            </div>
+            <div>
+              <MetaculusSource question={question} />
+            </div>
+          </>
+        ) : null}
+      </div>
+    );
+  }
+
+  return (
+    <div className="space-y-2 flex flex-col">
+      <GrayContainer title="Resizable preview">
+        <Resizable
+          minWidth={320}
+          bounds="window"
+          enable={{ right: true, left: true }}
+        >
+          <div ref={containerRef}>
+            <QuestionCard
+              container={(children) => (
+                <div className="px-4 py-3 bg-white">{children}</div>
+              )}
+              question={question}
+              showTimeStamp={true}
+              showExpandButton={false}
+              expandFooterToFullWidth={true}
+            />
          </div>
-          <div>
-            <ImageSource question={question} imgSrc={imgSrc} />
-          </div>
-          <div className="justify-self-stretch">
-            <MetaculusEmbed question={question} />
-          </div>
-          <div>
-            <MetaculusSource question={question} />
-          </div>
-        </>
-      ) : null}
+        </Resizable>
+      </GrayContainer>
+      <Button onClick={onCaptureButtonClick} size="small">
+        {mainButtonText}
+      </Button>
    </div>
  );
 };
--- a/src/web/questions/components/HistoryChart/InnerChart.tsx
+++ b/src/web/questions/components/HistoryChart/InnerChart.tsx
@ -13,9 +13,11 @@ import {
  VictoryVoronoiContainer,
 } from "victory";

-import { chartColors, ChartData, ChartSeries, height, width } from "./utils";
+import { chartColors, ChartData, ChartSeries, goldenRatio } from "./utils";

-let dateFormat = "MMM do y"; // "yyyy-MM-dd"
+const height = 200
+const width = 200 * goldenRatio
+let dateFormat = "dd/MM/yy"; // "yyyy-MM-dd" // "MMM do yy"

 // can't be replaced with React component, VictoryChart requires VictoryGroup elements to be immediate children
 const getVictoryGroup = ({
@ -37,7 +39,7 @@ const getVictoryGroup = ({
          data: {
            // strokeOpacity: highlight ?  1 : 0.5,
            strokeOpacity: highlight && !isBinary ? 0.8 : 0.6,
-            strokeWidth: highlight && !isBinary ? 4 : 3,
+            strokeWidth: highlight && !isBinary ? 2.5 : 1.5,
          },
        }}
      />
@ -71,9 +73,9 @@ export const InnerChart: React.FC<Props> = ({
  const domainMax =
    maxProbability < 0.5 ? Math.round(10 * (maxProbability + 0.05)) / 10 : 1;
  const padding = {
-    top: 20,
-    bottom: 75,
-    left: 70,
+    top: 12,
+    bottom: 33,
+    left: 30,
    right: 17,
  };

@ -99,12 +101,12 @@ export const InnerChart: React.FC<Props> = ({
                <VictoryLabel
                  style={[
                    {
-                      fontSize: 16,
+                      fontSize: 10,
                      fill: "black",
                      strokeWidth: 0.05,
                    },
                    {
-                      fontSize: 16,
+                      fontSize: 10,
                      fill: "#777",
                      strokeWidth: 0.05,
                    },
@ -118,7 +120,7 @@ export const InnerChart: React.FC<Props> = ({
                )}`
              }
              style={{
-                fontSize: 17, // needs to be set here and not just in labelComponent for text size calculations
+                fontSize: 10, // needs to be set here and not just in labelComponent for text size calculations
                fontFamily:
                  '"Gill Sans", "Gill Sans MT", "Seravek", "Trebuchet MS", sans-serif',
                // default font family from Victory, need to be specified explicitly for some reason, otherwise text size gets miscalculated
@ -128,10 +130,10 @@ export const InnerChart: React.FC<Props> = ({
                fill: "white",
              }}
              cornerRadius={4}
-              flyoutPadding={{ top: 4, bottom: 4, left: 16, right: 16 }}
+              flyoutPadding={{ top: 4, bottom: 4, left: 10, right: 10 }}
            />
          }
-          radius={50}
+          radius={20}
          voronoiBlacklist={
            [...Array(seriesList.length).keys()].map((i) => `line-${i}`)
            // see: https://github.com/FormidableLabs/victory/issues/545
@ -159,10 +161,10 @@ export const InnerChart: React.FC<Props> = ({
        }}
        tickLabelComponent={
          <VictoryLabel
-            dx={-40}
+            dx={-10}
            dy={0}
            angle={-30}
-            style={{ fontSize: 15, fill: "#777" }}
+            style={{ fontSize: 9, fill: "#777" }}
          />
        }
        scale={{ x: "time" }}
@ -174,7 +176,7 @@ export const InnerChart: React.FC<Props> = ({
          grid: { stroke: "#D3D3D3", strokeWidth: 0.5 },
        }}
        tickLabelComponent={
-          <VictoryLabel dy={0} style={{ fontSize: 18, fill: "#777" }} />
+          <VictoryLabel dy={0} dx={5} style={{ fontSize: 9, fill: "#777" }} />
        }
        // tickFormat specifies how ticks should be displayed
        tickFormat={(x) => `${x * 100}%`}
@ -205,6 +207,7 @@ export const InnerChart: React.FC<Props> = ({
            })
            */
      }
+      
    </VictoryChart>
  );
 };
--- a/src/web/questions/components/HistoryChart/index.tsx
+++ b/src/web/questions/components/HistoryChart/index.tsx
@ -23,7 +23,7 @@ export const HistoryChart: React.FC<Props> = ({ question }) => {
  const data = useMemo(() => buildChartData(question), [question]);

  return (
-    <div className="flex items-center flex-col space-y-4 sm:flex-row sm:space-y-0">
+    <div className="flex items-center space-y-4 sm:flex-row sm:space-y-0 ">
      <InnerChart data={data} highlight={highlight} />
      <Legend
        items={data.seriesNames.map((name, i) => ({
--- a/src/web/questions/components/HistoryChart/utils.ts
+++ b/src/web/questions/components/HistoryChart/utils.ts
@ -18,7 +18,7 @@ export const chartColors = [
  "#F59E0B", // amber-500
 ];

-const goldenRatio = (1 + Math.sqrt(5)) / 2;
+export const goldenRatio = (1 + Math.sqrt(5)) / 2;
 // used both for chart and for ssr placeholder
 export const width = 750;
 export const height = width / goldenRatio;
--- a/src/web/questions/components/PlatformLink.tsx
+++ b/src/web/questions/components/PlatformLink.tsx
@ -1,16 +1,6 @@
-import { FaExternalLinkAlt } from "react-icons/fa";
-
+import { BoxedLink } from "../../common/BoxedLink";
 import { QuestionFragment } from "../../fragments.generated";

 export const PlatformLink: React.FC<{ question: QuestionFragment }> = ({
  question,
-}) => (
-  <a
-    className="px-2 py-1 border-2 border-gray-400 rounded-lg text-black no-underline text-normal hover:bg-gray-100 flex flex-nowrap space-x-1 items-center"
-    href={question.url}
-    target="_blank"
-  >
-    <span>{question.platform.label}</span>
-    <FaExternalLinkAlt className="text-gray-400 inline sm:text-md text-md" />
-  </a>
-);
+}) => <BoxedLink url={question.url}>{question.platform.label}</BoxedLink>;
--- a/src/web/questions/components/QuestionCard/QuestionFooter.tsx
+++ b/src/web/questions/components/QuestionCard/QuestionFooter.tsx
@ -74,7 +74,7 @@ const getCurrencySymbolIfNeeded = ({
    "openInterest",
    "liquidity",
  ];
-  let dollarPlatforms = ["predictit", "kalshi", "polymarket"];
+  let dollarPlatforms = ["predictit", "kalshi", "polymarket", "insight"];
  if (indicatorsWhichNeedCurrencySymbol.includes(indicator)) {
    if (dollarPlatforms.includes(platform)) {
      return "$";
@ -172,6 +172,7 @@ export const QuestionFooter: React.FC<Props> = ({
      >
        {question.platform.label
          .replace("Good Judgment Open", "GJOpen")
+          .replace("Insight Prediction", "Insight")
          .replace(/ /g, "\u00a0")}
      </div>
      <div
--- a/src/web/questions/components/QuestionCard/index.tsx
+++ b/src/web/questions/components/QuestionCard/index.tsx
@ -1,7 +1,7 @@
 import Link from "next/link";
+import { ReactElement, ReactNode } from "react";
 import { FaExpand } from "react-icons/fa";
 import ReactMarkdown from "react-markdown";
-
 import { Card } from "../../../common/Card";
 import { CopyText } from "../../../common/CopyText";
 import { QuestionFragment } from "../../../fragments.generated";
@ -17,8 +17,8 @@ const truncateText = (length: number, text: string): string => {
    return text;
  }
  const breakpoints = " .!?";
-  let lastLetter: string | undefined = undefined;
-  let lastIndex: number | undefined = undefined;
+  let lastLetter
+  let lastIndex
  for (let index = length; index > 0; index--) {
    const letter = text[index];
    if (breakpoints.includes(letter)) {
@ -63,6 +63,7 @@ const LastUpdated: React.FC<{ timestamp: Date }> = ({ timestamp }) => (
 // Main component

 interface Props {
+  container?: (children: ReactNode) => ReactElement;
  question: QuestionFragment;
  showTimeStamp: boolean;
  expandFooterToFullWidth: boolean;
@ -71,6 +72,7 @@ interface Props {
 }

 export const QuestionCard: React.FC<Props> = ({
+  container = (children) => <Card>{children}</Card>,
  question,
  showTimeStamp,
  expandFooterToFullWidth,
@ -82,72 +84,70 @@ export const QuestionCard: React.FC<Props> = ({

  const isBinary = isQuestionBinary(question);

-  return (
-    <Card>
-      <div className="h-full flex flex-col space-y-4">
-        <div className="flex-grow space-y-4">
-          {showIdToggle ? (
-            <div className="mx-10">
-              <CopyText text={question.id} displayText={`[${question.id}]`} />
-            </div>
-          ) : null}
-          <div>
-            {showExpandButton ? (
-              <Link href={`/questions/${question.id}`} passHref>
-                <a className="float-right block ml-2 mt-1.5">
-                  <FaExpand
-                    size="18"
-                    className="text-gray-400 hover:text-gray-700"
-                  />
-                </a>
-              </Link>
-            ) : null}
-            <Card.Title>
-              <a
-                className="text-black no-underline"
-                href={question.url}
-                target="_blank"
-              >
-                {question.title}
+  return container(
+    <div className="h-full flex flex-col space-y-4">
+      <div className="flex-grow space-y-4">
+        {showIdToggle ? (
+          <div className="mx-10">
+            <CopyText text={question.id} displayText={`[${question.id}]`} />
+          </div>
+        ) : null}
+        <div>
+          {showExpandButton ? (
+            <Link href={`/questions/${question.id}`} passHref>
+              <a className="float-right block ml-2 mt-1.5">
+                <FaExpand
+                  size="18"
+                  className="text-gray-400 hover:text-gray-700"
+                />
              </a>
-            </Card.Title>
-          </div>
-          <div className={isBinary ? "flex justify-between" : "space-y-4"}>
-            <QuestionOptions question={question} maxNumOptions={5} />
-            <div className={`hidden ${showTimeStamp ? "sm:block" : ""}`}>
-              <LastUpdated timestamp={lastUpdated} />
-            </div>
-          </div>
-
-          {question.platform.id !== "guesstimate" && options.length < 3 && (
-            <div className="text-gray-500">
-              <DisplayMarkdown description={question.description} />
-            </div>
-          )}
-
-          {question.platform.id === "guesstimate" && question.visualization && (
-            <img
-              className="rounded-sm"
-              src={question.visualization}
-              alt="Guesstimate Screenshot"
-            />
-          )}
+            </Link>
+          ) : null}
+          <Card.Title>
+            <a
+              className="text-black no-underline"
+              href={question.url}
+              target="_blank"
+            >
+              {question.title}
+            </a>
+          </Card.Title>
        </div>
-        <div
-          className={`sm:hidden ${!showTimeStamp ? "hidden" : ""} self-center`}
-        >
-          {/* This one is exclusively for mobile*/}
-          <LastUpdated timestamp={lastUpdated} />
-        </div>
-        <div className="w-full">
-          <div className="mb-2 mt-1">
-            <QuestionFooter
-              question={question}
-              expandFooterToFullWidth={expandFooterToFullWidth}
-            />
+        <div className={isBinary ? "flex justify-between" : "space-y-4"}>
+          <QuestionOptions question={question} maxNumOptions={5} />
+          <div className={`hidden ${showTimeStamp ? "sm:block" : ""}`}>
+            <LastUpdated timestamp={lastUpdated} />
          </div>
        </div>
+
+        {question.platform.id !== "guesstimate" && options.length < 3 && (
+          <div className="text-gray-500">
+            <DisplayMarkdown description={question.description} />
+          </div>
+        )}
+
+        {question.platform.id === "guesstimate" && question.visualization && (
+          <img
+            className="rounded-sm"
+            src={question.visualization}
+            alt="Guesstimate Screenshot"
+          />
+        )}
+      </div>
+      <div
+        className={`sm:hidden ${!showTimeStamp ? "hidden" : ""} self-center`}
+      >
+        {/* This one is exclusively for mobile*/}
+        <LastUpdated timestamp={lastUpdated} />
+      </div>
+      <div className="w-full">
+        <div className="mb-2 mt-1">
+          <QuestionFooter
+            question={question}
+            expandFooterToFullWidth={expandFooterToFullWidth}
+          />
+        </div>
      </div>
-    </Card>
+    </div>
  );
 };
--- a/src/web/questions/components/QuestionOptions.tsx
+++ b/src/web/questions/components/QuestionOptions.tsx
@ -101,7 +101,7 @@ const OptionRow: React.FC<OptionProps> = ({ option, mode, textMode }) => {
      <div
        className={`flex-none rounded-md text-center ${
          mode === "primary"
-            ? "text-normal text-white px-2 py-0.5 font-bold"
+            ? "text-sm md:text-lg text-normal text-white px-2 py-0.5 font-bold"
            : "text-sm w-14 py-0.5"
        } ${
          mode === "primary"
@ -113,7 +113,7 @@ const OptionRow: React.FC<OptionProps> = ({ option, mode, textMode }) => {
      </div>
      <div
        className={`leading-snug ${
-          mode === "primary" ? "text-normal" : "text-sm"
+          mode === "primary" ? "text-sm md:text-lg text-normal" : "text-sm"
        } ${
          mode === "primary" ? textColor(option.probability) : "text-gray-700"
        }`}
--- a/src/web/questions/components/QuestionTitle.tsx
+++ b/src/web/questions/components/QuestionTitle.tsx
@ -10,7 +10,7 @@ export const QuestionTitle: React.FC<Props> = ({
  question,
  linkToMetaforecast,
 }) => (
-  <h1 className="sm:text-3xl text-xl">
+  <h1 className="sm:text-3xl text-lg">
    <a
      className="text-black no-underline hover:text-gray-700"
      href={
--- a/src/web/questions/components/Stars.tsx
+++ b/src/web/questions/components/Stars.tsx
@ -54,5 +54,5 @@ function getStarsColor(numstars: number) {
 }

 export const Stars: React.FC<{ num: number }> = ({ num }) => {
-  return <div className={getStarsColor(num)}>{getstars(num)}</div>;
+  return <div className={getStarsColor(num) + " text-xs md:text-lg"}>{getstars(num)}</div>;
 };
--- a/src/web/questions/pages/EmbedQuestionPage.tsx
+++ b/src/web/questions/pages/EmbedQuestionPage.tsx
@ -30,24 +30,25 @@ export const getServerSideProps: GetServerSideProps<Props> = async (
    props: {
      urqlState: ssrCache.extractData(),
      id,
+      question
    },
  };
 };

 const EmbedQuestionPage: NextPage<Props> = ({ id }) => {
  return (
-    <div className="bg-white min-h-screen">
+    <div className="block bg-white min-h-screen">
      <Query document={QuestionPageDocument} variables={{ id }}>
        {({ data: { result: question } }) =>
          question ? (
-            <div className="p-4">
-              <QuestionTitle question={question} linkToMetaforecast={true} />
+            <div className="flex flex-col p-2 w-full h-12/12">
+              {/*<QuestionTitle question={question} linkToMetaforecast={true} /> */}

-              <div className="mb-5 mt-5">
+              <div className="mb-1 mt-1">
                <QuestionInfoRow question={question} />
              </div>

-              <div className="mb-10">
+              <div className="mb-0">
                <QuestionChartOrVisualization question={question} />
              </div>
            </div>
--- a/src/web/questions/pages/QuestionPage.tsx
+++ b/src/web/questions/pages/QuestionPage.tsx
@ -1,11 +1,11 @@
 import { GetServerSideProps, NextPage } from "next";
 import NextError from "next/error";
+import React from "react";
 import ReactMarkdown from "react-markdown";
-
 import { Card } from "../../common/Card";
+import { Collapsible } from "../../common/Collapsible";
 import { CopyParagraph } from "../../common/CopyParagraph";
 import { Layout } from "../../common/Layout";
-import { LineHeader } from "../../common/LineHeader";
 import { Query } from "../../common/Query";
 import { QuestionWithHistoryFragment } from "../../fragments.generated";
 import { ssrUrql } from "../../urql";
@ -43,13 +43,51 @@ export const getServerSideProps: GetServerSideProps<Props> = async (
  };
 };

-const Section: React.FC<{ title: string }> = ({ title, children }) => (
-  <div className="space-y-2 flex flex-col items-start">
-    <h2 className="text-xl text-gray-900">{title}</h2>
+const Section: React.FC<{ title: string; id?: string }> = ({
+  title,
+  children,
+  id,
+}) => (
+  <div className="space-y-4 flex flex-col items-start" id={id}>
+    <div className="border-b-2 border-gray-200 w-full group">
+      <h2 className="text-xl leading-3 text-gray-900">
+        <span>{title}</span>
+        {id ? (
+          <>
+            {" "}
+            <a
+              className="text-gray-300 no-underline hidden group-hover:inline"
+              href={`#${id}`}
+            >
+              #
+            </a>
+          </>
+        ) : null}
+      </h2>
+    </div>
    <div>{children}</div>
  </div>
 );

+const EmbedSection: React.FC<{ question: QuestionWithHistoryFragment }> = ({
+  question,
+}) => {
+  const url = `https://${getBasePath()}/questions/embed/${question.id}`;
+  return (
+    <Section title="Embed" id="embed">
+      <CopyParagraph
+        text={`<iframe src="${url}" height="600" width="600" frameborder="0" />`}
+        buttonText="Copy HTML"
+      />
+      <div className="mt-2">
+        <Collapsible title="Preview">
+          {() => <iframe src={url} height="600" width="600" frameBorder="0" />}
+        </Collapsible>
+      </div>
+    </Section>
+  );
+};
+
 const LargeQuestionCard: React.FC<{
  question: QuestionWithHistoryFragment;
 }> = ({ question }) => {
@ -65,8 +103,8 @@ const LargeQuestionCard: React.FC<{
        <QuestionChartOrVisualization question={question} />
      </div>

-      <div className="mx-auto max-w-prose">
-        <Section title="Question description">
+      <div className="mx-auto max-w-prose space-y-8">
+        <Section title="Question description" id="description">
          <ReactMarkdown
            linkTarget="_blank"
            className="font-normal text-gray-900"
@ -74,39 +112,17 @@ const LargeQuestionCard: React.FC<{
            {question.description.replaceAll("---", "")}
          </ReactMarkdown>
        </Section>
-        <div className="mt-5">
-          <Section title="Indicators">
-            <IndicatorsTable question={question} />
-          </Section>
-        </div>
+        <Section title="Indicators" id="indicators">
+          <IndicatorsTable question={question} />
+        </Section>
+        <Section title="Capture" id="capture">
+          <CaptureQuestion question={question} />
+        </Section>
+        <EmbedSection question={question} />
      </div>
    </Card>
  );
 };
-const QuestionScreen: React.FC<{ question: QuestionWithHistoryFragment }> = ({
-  question,
-}) => (
-  <div className="space-y-8">
-    <LargeQuestionCard question={question} />
-    <div className="space-y-4">
-      <LineHeader>
-        <h1>Capture</h1>
-      </LineHeader>
-      <CaptureQuestion question={question} />
-      <LineHeader>
-        <h1>Embed</h1>
-      </LineHeader>
-      <div className="max-w-md mx-auto">
-        <CopyParagraph
-          text={`<iframe src="${
-            getBasePath() + `/questions/embed/${question.id}`
-          }" height="600" width="600" frameborder="0" />`}
-          buttonText="Copy HTML"
-        />
-      </div>
-    </div>
-  </div>
-);

 const QuestionPage: NextPage<Props> = ({ id }) => {
  return (
@ -115,7 +131,7 @@ const QuestionPage: NextPage<Props> = ({ id }) => {
        <Query document={QuestionPageDocument} variables={{ id }}>
          {({ data }) =>
            data.result ? (
-              <QuestionScreen question={data.result} />
+              <LargeQuestionCard question={data.result} />
            ) : (
              <NextError statusCode={404} />
            )
--- a/src/web/utils.ts
+++ b/src/web/utils.ts
@ -2,7 +2,7 @@ import { QuestionFragment } from "./fragments.generated";

 export const getBasePath = () => {
  if (process.env.NEXT_PUBLIC_VERCEL_URL) {
-    return `https://${process.env.NEXT_PUBLIC_VERCEL_URL}`;
+    return `https://metaforecast.org`;//`https://${process.env.NEXT_PUBLIC_VERCEL_URL}`;
  }

  // can be used for local development if you prefer non-default port
@ -14,8 +14,8 @@ export const getBasePath = () => {
 };

 export const cleanText = (text: string): string => {
-  // Note: should no longer be necessary?
-  // Still needed for e.g. /questions/rootclaim-what-caused-the-disappearance-of-malaysia-airlines-flight-370
+  // TODO - move to GraphQL:
+  // { description(clean: true, truncate: 250) }
  let textString = !!text ? text : "";
  textString = textString
    .replaceAll("] (", "](")
@ -23,12 +23,13 @@ export const cleanText = (text: string): string => {
    .replaceAll("( [", "([")
    .replaceAll(") ,", "),")
    .replaceAll("==", "") // Denotes a title in markdown
-    .replaceAll("Background\n", "")
-    .replaceAll("Context\n", "")
+    .replaceAll(/^#+\s+/gm, "")
+    .replaceAll(/^Background\n/gm, "")
+    .replaceAll(/^Context\n/gm, "")
    .replaceAll("--- \n", "- ")
    .replaceAll(/\[(.*?)\]\(.*?\)/g, "$1");
  textString = textString.slice(0, 1) == "=" ? textString.slice(1) : textString;
-  //console.log(textString)
+
  return textString;
 };

--- a/src/web/worker/searchWithAlgolia.ts
+++ b/src/web/worker/searchWithAlgolia.ts
@ -97,7 +97,7 @@ export default async function searchWithAlgolia({
        url: "https://metaforecast.org",
        platform: "metaforecast",
        platformLabel: "metaforecast",
-        description: "Maybe try a broader query?",
+        description: "Maybe try a broader query, e.g., reduce the number of 'stars' by clicking in 'Advanced options'?",
        options: [
          {
            name: "Yes",
@ -166,7 +166,7 @@ export default async function searchWithAlgolia({
      url: "https://metaforecast.org",
      platform: "metaforecast",
      platformLabel: "metaforecast",
-      description: "Maybe try a broader query? That said, we could be wrong.",
+      description: "Maybe try a broader query? Maybe try a broader query, e.g., reduce the number of 'stars' by clicking in 'Advanced options'? That said, we could be wrong.",
      options: [
        {
          name: "Yes",
--- a/src/web/worker/uploadToImgur.ts
+++ b/src/web/worker/uploadToImgur.ts
@ -5,7 +5,7 @@ export async function uploadToImgur(dataURL: string): Promise<string> {
    method: "post",
    url: "https://api.imgur.com/3/image",
    headers: {
-      Authorization: "Bearer 8e9666fb889318515a62208560d4e8393dac26d8",
+      Authorization: `Bearer ${process.env.IMGUR_BEARER}`,
    },
    data: {
      type: "base64",
--- a/yarn.lock
+++ b/yarn.lock
Author	SHA1	Message	Date
NunoSempere	5c9378833e	tweak: Add license, some small tweaks.	2023-01-05 18:24:26 +01:00
NunoSempere	5fc9930265	upgrade: safe dependencies	2022-11-29 09:10:30 +00:00
NunoSempere	685414e8fb	feat: upgrade dependencies part 1.	2022-11-29 09:00:31 +00:00
NunoSempere	fe4bba5169	tweak: change manifold quality indicators.	2022-11-26 13:27:00 +00:00
NunoSempere	e9796c545d	tweak: make indicators responsive to mobile	2022-11-17 21:58:15 +00:00
NunoSempere	fc84300712	tweak: change getBasePath implementation Unnecessary to expose the vercel url.	2022-11-17 20:16:24 +00:00
NunoSempere	73329df47b	feat: Fix embeds so that they are a bit more compressible. Also remove the title, which is variable width & thus very annoying to account for.	2022-11-17 20:12:31 +00:00
NunoSempere	dc1e75d99d	tweak: change embed page a bit more	2022-11-17 19:22:46 +00:00
NunoSempere	0a7d2d160a	tweak: Make embedded chart more compressible Per pointers here: <https://github.com/ForumMagnum/ForumMagnum/pull/6096#issuecomment-1317821272>	2022-11-17 18:19:24 +00:00
NunoSempere	67e4b825db	fix: bug	2022-11-09 22:15:29 +00:00
NunoSempere	dfe1de5279	feat: tweak metaforecast path in question embed. No idea why this works differently in that particular page.	2022-11-09 22:12:21 +00:00
NunoSempere	38a2fe8215	feat: add caching!!	2022-11-09 21:54:08 +00:00
NunoSempere	8ccb88558f	fix: yoga import bug	2022-11-09 21:37:32 +00:00
NunoSempere	31bfb357b3	fix: continue yoga-graphql update	2022-11-09 21:36:19 +00:00
NunoSempere	611d553193	feat: Start migration to yoga v3 Per <https://www.the-guild.dev/graphql/yoga-server/v3/migration/migration-from-yoga-v2>. This is necessay in order to use the caching functionality	2022-11-09 21:30:39 +00:00
NunoSempere	543ea966af	Revert "tweak: update squiggle endpoint" This reverts commit `f476d8d9ad`. It breaks the Google Sheets endpoint, for some reason.	2022-11-03 19:26:50 +00:00
NunoSempere	f476d8d9ad	tweak: update squiggle endpoint	2022-11-03 13:24:54 +00:00
NunoSempere	3f1334c02c	tweak: rename "contrib" to "scripts"	2022-11-03 13:11:11 +00:00
NunoSempere	78dd1dce40	tweak: modify readme	2022-11-02 11:55:17 +00:00
NunoSempere	a457d6dd50	tweak: Add item to to-do list.	2022-11-02 11:34:51 +00:00
NunoSempere	038b26ab7e	fix: refactor imgur bearer.	2022-11-02 11:33:27 +00:00
NunoSempere	a55f49d5a2	tweak: upgrade dependencies	2022-10-29 00:12:53 +01:00
NunoSempere	8a191bb694	tweak: Insight name display	2022-10-28 15:25:47 +01:00
NunoSempere	b9460be02d	tweak: symbol for insight markets volume in $$	2022-10-28 13:59:32 +01:00
NunoSempere	fd7839932d	tweak: console.log tweak	2022-10-28 13:59:14 +01:00
NunoSempere	1fa2aa1bdd	feat: insight prediction tweaks	2022-10-28 13:31:08 +01:00
NunoSempere	d739def318	feat: add insight non-binary markets	2022-10-28 13:25:58 +01:00
NunoSempere	bf89e4b11d	feat: save further insight progress.	2022-10-28 12:37:28 +01:00
NunoSempere	f5bf50456a	feat: Add insight markets So far restricted to: - Binary markets - Denominated in USD	2022-10-28 10:28:34 +01:00
NunoSempere	d7843b52c3	feat: save insight prediction changes	2022-10-27 23:35:44 +01:00
NunoSempere	370f332e64	tweak: add/clean to do items.	2022-10-27 19:12:30 +01:00
NunoSempere	c0a72b6b9c	fix: status spinner	2022-10-26 23:19:14 +01:00
Nuño Sempere	8c91993f78	Merge pull request #97 from quantified-uncertainty/fix-metaculus-validation Fix metaculus validation bug	2022-10-26 16:37:17 +01:00
Vyacheslav Matyukhin	2176c51d5f	community_prediction can be null on subquestions e.g. https://www.metaculus.com/api2/questions/12663/	2022-10-26 19:35:03 +04:00
NunoSempere	002a0e5e2f	tweak: Added some metaculus checking But this doesn't work, because many questions are being validated at once. Aarg.	2022-10-26 16:08:18 +01:00
NunoSempere	a873cc4497	feat: update manifold markets code.	2022-10-26 14:36:23 +01:00
NunoSempere	bc09456bb7	fix: hacky fix for typescript error	2022-10-26 13:53:11 +01:00
NunoSempere	fc9c222a44	fix: catch metaculus errors Current code isn't particularly resilient to API changes.	2022-10-26 13:44:14 +01:00
NunoSempere	83a01e6156	fix: Deal with <https://github.com/vercel/next.js/issues/8592 > Document this on the README, and add a few more to do items	2022-10-25 14:20:06 +02:00
NunoSempere	346e070d0e	fix: add typescript package	2022-10-25 13:24:52 +02:00
NunoSempere	2518707d6a	fix: Move to yarn npm wasn't dealing well with large numbers of dependencies, and the updating process is a bit clunky (requires me to delete the node_modules directory each time). Trying yarn for now	2022-10-25 13:17:11 +02:00
NunoSempere	ba84377eae	fix: update dependencies Metaforecast has been going down somewhat randomly. Hopefully that fixes some of it.	2022-10-22 22:20:13 +01:00
NunoSempere	f0f4188758	fix: remove netlify devdependencies No longer needed, since we are using vercel.	2022-10-21 13:22:58 +01:00
NunoSempere	a4b88e6023	fix: fix some outdated packages, I.	2022-10-21 13:20:43 +01:00
NunoSempere	e8f1839a95	fix: hopefully finally fix StaticImageData error	2022-10-21 13:14:09 +01:00
NunoSempere	63628c96fe	Revert "fix: type error" This reverts commit `ed0c6e0588`.	2022-10-21 13:11:49 +01:00
NunoSempere	0d84c26e08	Revert "fix: try another fix for StaticImageData" This reverts commit `1bf1cf9c83`.	2022-10-21 13:11:12 +01:00
NunoSempere	1bf1cf9c83	fix: try another fix for StaticImageData	2022-10-21 13:04:21 +01:00
NunoSempere	ed0c6e0588	fix: type error See: <https://github.com/vercel/next.js/issues/29788>	2022-10-21 12:57:53 +01:00
NunoSempere	133db36d69	fix: autoprefixer error, node location error See: - <https://github.com/twbs/bootstrap/issues/36259> - <https://nextjs.org/docs/messages/nested-middleware>	2022-10-21 12:53:03 +01:00
NunoSempere	9766b49046	fix: update graphql dependencies.	2022-10-21 12:41:02 +01:00
NunoSempere	a78918da61	upgrade: graphql packages	2022-10-09 13:11:48 +01:00
NunoSempere	17ba3c1ce4	Revert "yolo: update all dependencies" This reverts commit `c48a80b0d2`.	2022-10-09 13:06:17 +01:00
NunoSempere	c48a80b0d2	yolo: update all dependencies	2022-10-09 13:05:12 +01:00
NunoSempere	aff30ac0c4	fix: insight types	2022-10-09 12:52:01 +01:00
NunoSempere	0cddbf69b0	fix: goodjudgmentopen type	2022-10-09 12:47:25 +01:00
NunoSempere	e4a3f38ddf	Merge branch 'master' of github.com:quantified-uncertainty/metaforecast	2022-10-09 12:42:09 +01:00
NunoSempere	0e3cede352	Revert "feat: save Insight Prediction progress" This reverts commit `3892db1157`.	2022-10-09 12:41:16 +01:00
NunoSempere	aeece45756	Revert "feat: save Insight Prediction progress" This reverts commit `3892db1157`. It should make graphql fixing easier	2022-10-09 12:30:24 +01:00
NunoSempere	3892db1157	feat: save Insight Prediction progress Note that the fetcher is subject to this bug: <https://github.com/quantified-uncertainty/metaforecast/issues/94> which means that it'll hit insight pretty hard.	2022-07-28 14:05:35 -04:00
NunoSempere	69ab8f905c	feat: wrange Insight Prediction API	2022-07-28 13:33:40 -04:00
NunoSempere	a751caf8fb	tweak: Hide Insight Prediction code for now Asking the guy for less rate limiting in my api access.	2022-07-08 22:56:56 -04:00
NunoSempere	494e8d74ad	feat: Fix Good Judgment flow, start writting the Insight Parser. Note that the Insight Prediction parser is in fact not complete	2022-07-08 22:25:11 -04:00
NunoSempere	cb6f9239bf	tweak: Update README.md	2022-07-08 20:42:31 -04:00
NunoSempere	c9bfba8474	fix: Good Judgment workflow	2022-07-08 17:50:15 -04:00
NunoSempere	acc85dad63	fix: update squiggle version	2022-06-18 12:46:01 -04:00
Vyacheslav Matyukhin	f98958f27e	Merge pull request #92 from quantified-uncertainty/better-cleantext Fix cleanText for metaculus	2022-06-05 00:42:04 +03:00
Vyacheslav Matyukhin	7a5b7837f5	fix: better heading cleanups	2022-06-05 00:38:16 +03:00
Vyacheslav Matyukhin	1a0e42147e	feat: cleanText improvements	2022-06-05 00:34:24 +03:00
Vyacheslav Matyukhin	00615c1e63	feat: delete questions from frontpage on deletion	2022-06-03 20:03:13 +03:00
Vyacheslav Matyukhin	ceeeff9681	fix: metaculus on groups	2022-06-03 16:54:45 +03:00
Vyacheslav Matyukhin	5bf24a58ef	feat: metaculus fetcher skips unknown types instead of failing	2022-06-03 13:30:17 +03:00
Vyacheslav Matyukhin	646397d8d4	fix: handle empty metaculus prediction	2022-06-03 12:01:02 +03:00
Vyacheslav Matyukhin	2345b81350	feat: support "discussion" metaculus type	2022-06-03 11:43:35 +03:00
Vyacheslav Matyukhin	de47956e29	feat: tune SLEEP_TIME for metaculus	2022-06-03 11:27:18 +03:00
Vyacheslav Matyukhin	c96627e2cf	feat: skip metaculus questions earlier	2022-06-03 11:13:57 +03:00
Vyacheslav Matyukhin	cc2257b626	feat: collapsible embed preview instead of button	2022-06-01 23:31:41 +03:00
Vyacheslav Matyukhin	c051f0dc7d	Merge pull request #90 from quantified-uncertainty/embed-question-page-2 Embed question page 2	2022-06-01 00:34:53 +03:00
Vyacheslav Matyukhin	c1330101e1	fix: types	2022-06-01 00:24:27 +03:00
Vyacheslav Matyukhin	558c0964e4	feat: section anchors	2022-06-01 00:20:29 +03:00
Vyacheslav Matyukhin	6e25f8bcd2	feat: question page style changes	2022-06-01 00:20:29 +03:00
Vyacheslav Matyukhin	69e5a6ece5	feat: better question page layout (WIP)	2022-06-01 00:20:29 +03:00
Vyacheslav Matyukhin	8fac309bf3	Merge pull request #88 from quantified-uncertainty/embed-question-page Embed question page	2022-06-01 00:12:21 +03:00
Vyacheslav Matyukhin	c3d144337b	Merge pull request #84 from quantified-uncertainty/metaculus-improvements Metaculus fetcher improvements	2022-05-31 22:11:09 +03:00
Vyacheslav Matyukhin	f5a3c16322	feat: metaculus group questions, description from api	2022-05-30 23:39:13 +03:00
Vyacheslav Matyukhin	d684d074f5	feat: metaculus fetcher takes markdown description from js vars	2022-05-19 14:16:08 +04:00
Vyacheslav Matyukhin	4d736f711d	feat: metaculus validates api, supports --id cli arg	2022-05-19 14:16:08 +04:00