manifold/functions
James Grugett 06b7e49e98
[In progress] Server-side feed computation (#106)
* Store view counts & last viewed time

* Schedule updating user recommendations. Compute using tf-idf.

* Update contract's lastBetTime and lastCommentTime on new bets and comments.

* Remove contract's lastUpdatedTime

* Remove folds activity feed

* Implement getFeed cloud function

* Hook up client to use getFeed

* Script to cache viewCounts and lastViewTime

* Batched wait all userRecommendations

* Cache view script runs on all users

* Update user feed each hour and get feed from cache doc.

* Delete view cache script

* Update feed script

* Tweak feed algorithm

* Compute recommendation scores from updateUserFeed

* Disable lastViewedScore factor

* Update lastCommentTime script

* Comment out console.log

* Fix timeout issue by calling new cloud functions with part of the work.

* Listen for contract updates to feed.

* Handle new user: use default feed of top markets this week

* Track lastUpdatedTime

* Tweak logic of calling cloud functions in batches

* Tweak cloud function batching
2022-05-01 11:36:54 -05:00
..
src [In progress] Server-side feed computation (#106) 2022-05-01 11:36:54 -05:00
.gitignore Separate free response answers & comments (#100) 2022-04-26 07:24:57 -06:00
package.json Support local firestore emulation (#91) 2022-04-25 09:46:35 -06:00
README.md Readme: must be in dev for emulators to work 2022-04-28 12:29:02 -06:00
tsconfig.json Folds (#34) 2022-01-21 17:21:46 -06:00

NOTE: Adapted from One Word's /functions doc. Fix any errors you see!

Firestore Cloud Functions

This is code that doesn't make sense on the frontend client, e.g.

  • Long-running or slow operation (database)
  • Tasks that need to be run every so often (syncing email list to Mailjet)
  • Anything we should't trust to clients (secrets, auth)

If you want to make and test changes, you'll have to do a bit of setup...

Installing

Adapted from https://firebase.google.com/docs/functions/get-started

  1. $ cd functions to switch to this folder
  2. $ yarn global add firebase-tools to install the Firebase CLI globally
  3. $ yarn to install JS dependencies
  4. $ firebase login to authenticate the CLI tools to Firebase
  5. $ firebase use dev to choose the dev project

For local development

  1. $ firebase functions:config:get > .runtimeconfig.json to cache secrets for local dev
  2. Install gcloud CLI
  3. $ brew install java to install java if you don't already have it
    1. $ echo 'export PATH="/usr/local/opt/openjdk/bin:$PATH"' >> ~/.zshrc to add java to your path
  4. $ gcloud auth login to authenticate the CLI tools to Google Cloud
  5. $ gcloud config set project <project-id> to choose the project ($ gcloud projects list to see options)
  6. $ mkdir firestore_export to create a folder to store the exported database
  7. $ yarn db:update-local-from-remote to pull the remote db from Firestore to local
    1. TODO: this won't work when open source, we'll have to point to the public db

Developing locally

  1. $ firebase use dev if you haven't already
  2. $ yarn serve to spin up the emulators
    1. The Emulator UI is at http://localhost:4000; the functions are hosted on :5001. Note: You have to kill and restart emulators when you change code; no hot reload =(
  3. $ yarn dev:emulate in /web to connect to emulators with the frontend
    1. Note: emulated database is cleared after every shutdown

Firestore Commands

  • db:update-local-from-remote - Pull the remote db from Firestore to local, also calls:
    • db:backup-remote - Exports the remote dev db to the backup folder on Google Cloud Storage (called on every db:update-local-from-remote)
    • db:rename-remote-backup-folder - Renames the remote backup folder (called on every db:backup-remote to preserve the previous db backup)
  • db:backup-local - Save the local db changes to the disk (overwrites existing)

Debugging

  • Find local logs directly in the shell that ran $ yarn dev
  • Find deployed logs here

Deploying

  1. $ firebase use prod to switch to prod
  2. $ yarn deploy to push your changes live! (Future TODO: auto-deploy functions on Git push)

Secrets management

Secrets are strings that shouldn't be checked into Git (eg API keys, passwords). We store these using environment config on Firebase Functions. Some useful workflows:

  • Set a secret: $ firebase functions:config:set stripe.test_secret="THE-API-KEY"
  • Preview all secrets: $ firebase functions:config:get
  • Cache for local dev:$ firebase functions:config:get > .runtimeconfig.json