Compare commits

...

3 Commits

30 changed files with 49 additions and 73 deletions

View File

@ -29,7 +29,7 @@ The name of this repository is a pun on two meanings of "time to": "how much tim
| Nim | 40.8ms | 84 | | Nim | 40.8ms | 84 |
| Lua (LuaJIT) | 69.9ms | 82 | | Lua (LuaJIT) | 69.9ms | 82 |
| OCaml (flambda) | 187.9ms | 123 | | OCaml (flambda) | 187.9ms | 123 |
| Squiggle (bun) | 0.386s | 14* | | Squiggle (bun) | 0.387s | 14* |
| Javascript (node) | 0.445s | 69 | | Javascript (node) | 0.445s | 69 |
| SquigglePy (v0.27) | 1.507s | 18* | | SquigglePy (v0.27) | 1.507s | 18* |
| R (3.6.1) | 4.508s | 49 | | R (3.6.1) | 4.508s | 49 |
@ -77,17 +77,23 @@ Although the above paragraphs were written in the first person, the C code was w
### squiggle.c ### squiggle.c
squiggle.c is a minimalistic library focused on understandability and being self-contained. It grew from the initial C code in this repository. You can see the code for the library [here](https://git.nunosempere.com/personal/squiggle.c), and the code for the example we are discussing [here](https://git.nunosempere.com/personal/squiggle.c/src/branch/master/examples/02_many_samples_time_to_botec). squiggle.c is a minimalistic library focused on understandability and being self-contained. I've put a bunch of thought into how to design this in a way which is clean and fast. It grew from the initial C code in this repository. You can see the code for the library [here](https://git.nunosempere.com/personal/squiggle.c), which contains a thoughful README which I recommend people doing Monte Carlo estimation stuff read.
I like the [operator](http://duskos.org/#operator) section of [Dusk OS](http://duskos.org/):
> Dusk OS doesn't have users, but operators. What's the difference? Control. You use a phone, you use a coffee machine, hell you even use a car these days. But you operate a bulldozer, you operate a crane, you operate a plane.
### NodeJS and Squiggle ### NodeJS and Squiggle
Using [bun](https://bun.sh/) instead of node is actually a bit slower. Also, both the NodeJS and the Squiggle code use [stdlib](https://stdlib.io/) in their innards, which has a bunch of interleaved functions that make the code slower. It's possible that not using that external library could make the code faster. But at the same time, the js approach does seem to be to use external libraries whenever possible. Using [bun](https://bun.sh/) instead of node is actually a bit slower for the raw js code. Also, both the NodeJS and the Squiggle code use [stdlib](https://stdlib.io/) in their innards, which has a bunch of interleaved functions that make the code slower. It's possible that not using that external library could make the code faster. But at the same time, the js approach does seem to be to use external libraries whenever possible.
I am not particularly sure that the Squiggle code is actually producing 1M samples, but I am also not in a rush to debug this. In general, Squiggle tries to present a simple interface to the user, leading to "hiding the magic" and having a bunch of [bugs](https://github.com/quantified-uncertainty/squiggle/labels/Bug), whereas I think the right tradeoff for me is to have some simple interface that I can operate skillfully (i.e., squiggle.c).
### Python ### Python and Squigglepy
For the Python code, it's possible that the lack of speed is more a function of me not being as familiar with Python. It's also very possible that the code would run faster with [PyPy](https://doc.pypy.org). For the Python code, it's possible that the lack of speed is more a function of me not being as familiar with Python. It's also very possible that the code would run faster with [PyPy](https://doc.pypy.org).
In terms of complexity, SquigglePy seems to be between squiggle.c and the original squiggle. Like the original suqiggle, it also hides its stuff behind semi-magic wrappers, leading to e.g. ambiguities like around [correlated samples](https://git.nunosempere.com/personal/squiggle.c#correlated-samples) and generally having moving pieces that I don't understand. On the other hand, the code *is* short enough so that one person could read it in a few afternoons and roughly understand it. In terms of speed, SquigglePy seems slow.
### R ### R

View File

@ -1,5 +1,8 @@
OUTPUT=./samples OUTPUT=./samples
build:
gcc -O3 samples.c ./squiggle_c/squiggle.c ./squiggle_c/squiggle_more.c -lm -fopenmp -o $(OUTPUT)
install: install:
rm -r squiggle_c rm -r squiggle_c
git clone https://git.nunosempere.com/personal/squiggle.c git clone https://git.nunosempere.com/personal/squiggle.c
@ -8,9 +11,17 @@ install:
cp -r squiggle_c/examples/more/12_time_to_botec_parallel/example.c samples.c cp -r squiggle_c/examples/more/12_time_to_botec_parallel/example.c samples.c
sed -i 's|../../..|squiggle_c|' samples.c sed -i 's|../../..|squiggle_c|' samples.c
build:
gcc -O3 samples.c ./squiggle_c/squiggle.c ./squiggle_c/squiggle_more.c -lm -fopenmp -o $(OUTPUT)
time-linux: time-linux:
@echo "Running 100x and taking avg time: OMP_NUM_THREADS=16 $(OUTPUT)" @echo "Running 100x and taking avg time: OMP_NUM_THREADS=16 $(OUTPUT)"
@t=$$(/usr/bin/time -f "%e" -p bash -c 'for i in {1..100}; do OMP_NUM_THREADS=16 $(OUTPUT); done' 2>&1 >/dev/null | grep real | awk '{print $$2}' ); echo "scale=2; 1000 * $$t / 100" | bc | sed "s|^|Time using 16 threads: |" | sed 's|$$|ms|' && echo @t=$$(/usr/bin/time -f "%e" -p bash -c 'for i in {1..100}; do OMP_NUM_THREADS=16 $(OUTPUT); done' 2>&1 >/dev/null | grep real | awk '{print $$2}' ); echo "scale=2; 1000 * $$t / 100" | bc | sed "s|^|Time using 16 threads: |" | sed 's|$$|ms|' && echo
install-small:
rm -r squiggle_c
git clone https://git.nunosempere.com/personal/squiggle.c
mv squiggle.c squiggle_c
sudo rm -r squiggle_c/.git
cp -r squiggle_c/examples/core/02_time_to_botec/example.c samples.c
sed -i 's|../../..|squiggle_c|' samples.c
build-small:
gcc -O3 samples.c ./squiggle_c/squiggle.c -lm -o $(OUTPUT)

View File

@ -248,18 +248,18 @@ make tidy
It emits one warning about something I already took care of, so by default I've suppressed it. I think this is good news in terms of making me more confident that this simple library is correct :). It emits one warning about something I already took care of, so by default I've suppressed it. I think this is good news in terms of making me more confident that this simple library is correct :).
### Division between core functions and extraneous expansions ### Division between core functions and squiggle_moreneous expansions
This library differentiates between core functions, which are pretty tightly scoped, and expansions and convenience functions, which are more meandering. Expansions are in `extra.c` and `extra.h`. To use them, take care to link them: This library differentiates between core functions, which are pretty tightly scoped, and expansions and convenience functions, which are more meandering. Expansions are in `squiggle_more.c` and `squiggle_more.h`. To use them, take care to link them:
``` ```
// In your C source file // In your C source file
#include "extra.h" #include "squiggle_more.h"
``` ```
``` ```
# When compiling: # When compiling:
gcc -std=c99 -Wall -O3 example.c squiggle.c extra.c -lm -o ./example gcc -std=c99 -Wall -O3 example.c squiggle.c squiggle_more.c -lm -o ./example
``` ```
@ -290,6 +290,13 @@ Behaviour on error can be toggled by the `EXIT_ON_ERROR` variable. This library
Overall, I'd describe the error handling capabilities of this library as pretty rudimentary. For example, this program might fail in surprising ways if you ask for a lognormal with negative standard deviation, because I haven't added error checking for that case yet. Overall, I'd describe the error handling capabilities of this library as pretty rudimentary. For example, this program might fail in surprising ways if you ask for a lognormal with negative standard deviation, because I haven't added error checking for that case yet.
## Extra: confidence intervals
// to do
## Extra paralellism
// to do
## Related projects ## Related projects

View File

@ -1,5 +1,4 @@
#include "../../../squiggle.h" #include "../../../squiggle.h"
#include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>

View File

@ -1,5 +1,4 @@
#include "../../../squiggle.h" #include "../../../squiggle.h"
#include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>

View File

@ -1,29 +1,7 @@
#include "../../../squiggle.h" #include "../../../squiggle.h"
#include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>
// Estimate functions
double sample_0(uint64_t* seed)
{
return 0;
}
double sample_1(uint64_t* seed)
{
return 1;
}
double sample_few(uint64_t* seed)
{
return sample_to(1, 3, seed);
}
double sample_many(uint64_t* seed)
{
return sample_to(2, 10, seed);
}
int main() int main()
{ {
// set randomness seed // set randomness seed
@ -34,6 +12,11 @@ int main()
double p_b = 0.5; double p_b = 0.5;
double p_c = p_a * p_b; double p_c = p_a * p_b;
double sample_0(uint64_t* seed){ return 0; }
double sample_1(uint64_t* seed) { return 1; }
double sample_few(uint64_t* seed) { return sample_to(1, 3, seed); }
double sample_many(uint64_t* seed) { return sample_to(2, 10, seed); }
int n_dists = 4; int n_dists = 4;
double weights[] = { 1 - p_c, p_c / 2, p_c / 4, p_c / 4 }; double weights[] = { 1 - p_c, p_c / 2, p_c / 4, p_c / 4 };
double (*samplers[])(uint64_t*) = { sample_0, sample_1, sample_few, sample_many }; double (*samplers[])(uint64_t*) = { sample_0, sample_1, sample_few, sample_many };
@ -45,11 +28,5 @@ int main()
} }
printf("Mean: %f\n", array_mean(result_many, n_samples)); printf("Mean: %f\n", array_mean(result_many, n_samples));
// printf("result_many: [");
// for(int i=0; i<100; i++){
// printf("%.2f, ", result_many[i]);
// }
// printf("]\n");
free(seed); free(seed);
} }

View File

@ -1,5 +1,4 @@
#include "../../../squiggle.h" #include "../../../squiggle.h"
#include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>

View File

@ -1,5 +1,4 @@
#include "../../../squiggle.h" #include "../../../squiggle.h"
#include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>

View File

@ -1,5 +1,4 @@
#include "../../../squiggle.h" #include "../../../squiggle.h"
#include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>

View File

@ -1,6 +1,5 @@
#include "../../../squiggle.h" #include "../../../squiggle.h"
#include "../../../squiggle_more.h" #include "../../../squiggle_more.h"
#include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>

View File

@ -1,7 +1,6 @@
#include "../../../squiggle.h" #include "../../../squiggle.h"
#include "../../../squiggle_more.h" #include "../../../squiggle_more.h"
#include <math.h> #include <math.h>
#include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>
#include <time.h> #include <time.h>

View File

@ -1,7 +1,6 @@
#include "../../../squiggle.h" #include "../../../squiggle.h"
#include "../../../squiggle_more.h" #include "../../../squiggle_more.h"
#include <math.h> #include <math.h>
#include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>
#include <time.h> #include <time.h>

View File

@ -1,6 +1,5 @@
#include "../../../squiggle.h" #include "../../../squiggle.h"
#include "../../../squiggle_more.h" #include "../../../squiggle_more.h"
#include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>

View File

@ -1,7 +1,6 @@
#include "../../../squiggle.h" #include "../../../squiggle.h"
#include "../../../squiggle_more.h" #include "../../../squiggle_more.h"
#include <math.h> #include <math.h>
#include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>

View File

@ -1,7 +1,6 @@
#include "../../../squiggle.h" #include "../../../squiggle.h"
#include "../../../squiggle_more.h" #include "../../../squiggle_more.h"
#include <math.h> #include <math.h>
#include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>

View File

@ -1,7 +1,6 @@
#include "../../../squiggle.h" #include "../../../squiggle.h"
#include "../../../squiggle_more.h" #include "../../../squiggle_more.h"
#include <math.h> #include <math.h>
#include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>

View File

@ -1,7 +1,6 @@
#include "../../../squiggle.h" #include "../../../squiggle.h"
#include "../../../squiggle_more.h" #include "../../../squiggle_more.h"
#include <math.h> #include <math.h>
#include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>

View File

@ -1,7 +1,6 @@
#include "../../../squiggle.h" #include "../../../squiggle.h"
#include "../../../squiggle_more.h" #include "../../../squiggle_more.h"
#include <math.h> #include <math.h>
#include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>

View File

@ -1,7 +1,6 @@
#include "../../../squiggle.h" #include "../../../squiggle.h"
#include "../../../squiggle_more.h" #include "../../../squiggle_more.h"
#include <math.h> #include <math.h>
#include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>

View File

@ -1,6 +1,5 @@
#include "../../../squiggle.h" #include "../../../squiggle.h"
#include "../../../squiggle_more.h" #include "../../../squiggle_more.h"
#include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>

View File

@ -1,6 +1,5 @@
#include "../../../squiggle.h" #include "../../../squiggle.h"
#include "../../../squiggle_more.h" #include "../../../squiggle_more.h"
#include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>

View File

@ -11,9 +11,6 @@ time-bun:
run-bun: run-bun:
bun src/samples.js bun src/samples.js
patch:
sed -i 's/defaultSampleCount: 1000/defaultSampleCount: 1000000/g' node_modules/@quri/squiggle-lang/src/magicNumbers.ts node_modules/@quri/squiggle-lang/dist/magicNumbers.js
run-node: run-node:
node src/samples.js node src/samples.js

View File

@ -1,7 +0,0 @@
I can't currently figure out how to change the number of samples from within squiggle for mixtures,
so instead I monkey patched it
- https://github.com/quantified-uncertainty/squiggle/issues/2560
- grep -r . -e defaultSampleCount
- sed -i 's/defaultSampleCount: 1000/defaultSampleCount: 1000000/g' node_modules/@quri/squiggle-lang/src/magicNumbers.ts node_modules/@quri/squiggle-lang/dist/magicNumbers.js

View File

@ -1,7 +1,7 @@
export const epsilon_float = 2.22044604925031308e-16; export const epsilon_float = 2.22044604925031308e-16;
export const Environment = { export const Environment = {
defaultXYPointLength: 1000, defaultXYPointLength: 1000,
defaultSampleCount: 1000000, defaultSampleCount: 1000,
sparklineLength: 20, sparklineLength: 20,
}; };
export const OpCost = { export const OpCost = {

View File

@ -2,7 +2,7 @@ export const epsilon_float = 2.22044604925031308e-16; // via pervasives.js
export const Environment = { export const Environment = {
defaultXYPointLength: 1000, defaultXYPointLength: 1000,
defaultSampleCount: 1000000, defaultSampleCount: 1000,
sparklineLength: 20, sparklineLength: 20,
}; };

View File

@ -19,9 +19,11 @@ mean(result)
async function main(){ async function main(){
let output = await run(squiggle_code, { let output = await run(squiggle_code, {
defaultXYPointLength: 1000000, environment: {
defaultSampleCount: 1000000, xyPointLength: 1000000,
sparklineLength: 20, sampleCount: 1000000,
sparkLine: 20,
}
}) })
console.log(output.value.result.value) console.log(output.value.result.value)
} }