7.0 KiB
A calculator for distributions, for Fermi estimation
This project is a minimalist, calculator-style DSL for fermi estimation. It can multiply, divide, add and substract scalars, lognormals and beta distributions, and supports variables.
Motivation
Sometimes, Squiggle, simple squiggle or squiggle.c are still too complicated and un-unix-like. In particular, their startup cost is not instant.
Installation
make build
sudo make install
fermi
Usage
$ fermi
5000000 12000000
=> 5.0M 12.0M
* beta 1 200
=> 1.9K 123.1K
* 30 180
=> 122.9K 11.7M
/ 48 52
=> 2.5K 234.6K
/ 5 6
=> 448.8 43.0K
/ 6 8
=> 64.5 6.2K
/ 60
=> 1.1 103.7
Perhaps this example is more understandable with comments and better units:
$ fermi
5M 12M # number of people living in Chicago
beta 1 200 # fraction of people that have a piano
30 180 # minutes it takes to tune a piano, including travel time
/ 48 52 # weeks a year that piano tuners work for
/ 5 6 # days a week in which piano tuners work
/ 6 8 # hours a day in which piano tuners work
/ 60 # minutes to an hour
=: piano_tuners
If you type "help" (or run fermi -h), you can see a small grammar and some optional command flags:
$ fermi
1. Grammar:
Operation | Variable assignment | Special
Operation: operator operand
operator: (empty) | * | / | + | -
operand: scalar | lognormal | beta | variable
lognormal: low high
beta: beta alpha beta
Variable assignment: =: variable_name
Variable assignment and clear stack: =. variable_name
Suffixes: %, K, M, B, T
Special commands:
Comment: # this is a comment
Summary stats: stats
Clear stack: clear | c | .
Print debug info: debug | d
Print help message: help | h
Start additional stack: operator (
Return from additional stack )
Exit: exit | e
Examples:
+ 2
/ 2.5
* 1 10 (interpreted as lognormal)
+ 1 10
* beta 1 10
1 10 (multiplication taken as default operation)
=: x
.
1 100
+ x
# this is a comment
* 1 12 # this is an operation followed by a comment
* (
1 10
+ beta 1 100
)
exit
Command flags:
-echo
Specifies whether inputs should be echoed back. Useful if reading from a file
. -f string
Specifies a file with a model to run. Sets the echo command to true by default.
-n int
Specifies the number of samples to draw when using samples (default 100000)
-h Shows help message
You can see real life examples here, here, here, here, here, here
Tips & tricks
- It's conceptually clearer to have all the multiplications first and then all the divisions
- For things between 0 and 1, consider using a beta distribution
Command line options
You can specify the number of samples to draw when algebraic manipulations are not sufficient:
$ fermi -n 1000000
$ fermi -n 1_000_000
You also run a file with the -f option
$ fermi -f more/piano-tuners.fermi
Integrations with linux utilities
Because the model reads from standard input, you can a model to it:
$ cat more/piano-tuners.fermi | fermi
In that case, you will probably want to use the echo flag as well
$ cat more/piano-tuners-commented.fermi | fermi -echo
You can make a model an executable file by running $ chmod -x model.fermi and then adding the following at the top!
#!/bin/usr/fermi -f
You can save a session to a logfile with tee:
fermi | tee -a fermi.log
Different levels of complexity
The top level f.go file (420 lines) has a bunch of complexity: variables, parenthesis, samples, beta distributions, number of samples, etc. In the simple/ folder:
- f_simple.go (370 lines) strips variables and parenthesis, but keeps beta distributions, samples, and addition and substraction
- f_minimal.go (140 lines) strips everything that isn't lognormal and scalar multiplication and addition, plus a few debug options.
Roadmap
Done:
- Write README
- Add division?
- Read from file?
- Save to file?
- Allow comments?
- Use a sed filter?
- Add proper comment processing
- Add show more info version
- Scalar multiplication and division
- Think how to integrate with squiggle.c to draw samples
- Copy the time to botec go code
- Define samplers
- Call those samplers when operating on distributions that can't be operted on algebraically
- Display output more nicely, with K/M/B/T
- Consider the following: make this into a stack-based DSL, with:
- Variables that can be saved to and then displayed
- Other types of distributions, particularly beta distributions? => But then this requires moving to bags of samples. It could still be ~instantaneous though.
- Added bags of samples to support addition and multiplication of betas and lognormals
- Figure out go syntax for
- Maps
- Joint types
- Enums
- Fix correlation problem, by spinning up a new randomness thing every time some serial computation is done.
- Clean up error code. Right now only needed for division
- Maintain both a more complex thing that's more featureful and the more simple multiplication of lognormals thing.
- Allow input with K/M/T
- Document parenthesis syntax
- Specify number of samples as a command line option
- Figure out how to make models executable, by adding a #!/bin/bash-style command at the top?
- Make -n flag work
- Add flag to repeat input lines (useful when reading from files)
- Add percentages
To (possibly) do:
- Fix lognormal multiplication and division by 0 or < 0
- Consider adding an understanding of percentages
- With the -f command line option, the program doesn't read from stdin after finishing reading the file
- Add functions. Now easier to do with an explicit representation of the stakc
- Think about how to draw a histogram from samples
- Dump samples to file
- Represent samples/statistics in some other way
- Perhaps use qsort rather than full sorting
- Program into a small device, like a calculator?
- Units?
Discarded:
Think of some way of calling bc