add LuaJIT timing, redo nim, add warning about timings

This commit is contained in:
NunoSempere 2023-06-10 19:31:13 -06:00
parent c9eeaf5b5d
commit 541aed0c97
4 changed files with 82 additions and 14 deletions

View File

@ -34,15 +34,19 @@ The title of this repository is a pun on two meanings of "time to": "how much ti
| Language | Time | Lines of code |
|-----------------------------|-----------|---------------|
| C (optimized, 16 threads) | 5ms | 249 |
| Nim | 68ms | 84 |
| Nim | 38ms | 84 |
| Lua (LuaJIT) | 68ms | 82 |
| Lua | 278ms | 82 |
| C (naïve implementation) | 292ms | 149 |
| Javascript (NodeJS) | 732ms | 69 |
| Squiggle | 1,536s | 14 |
| SquigglePy | 1.602s | 18 |
| Squiggle | 1,536s | 14* |
| SquigglePy | 1.602s | 18* |
| R | 7,000s | 49 |
| Python (CPython) | 16,641s | 56 |
Time measurements taken with the [time](https://man7.org/linux/man-pages/man1/time.1.html) tool, using 1M samples.
Time measurements taken with the [time](https://man7.org/linux/man-pages/man1/time.1.html) tool, using 1M samples. But different implementations use different algorithms and, occasionally, different time measuring methodologies (for the C, Nim and Lua implementations, I run the program 100 times and take the mean). Their speed was also measured under different loads in my machine. So I think that these time estimates are accurate within maybe ~2x or so.
Note that the number of lines is much shorter for Squiggle and SquigglePy because I'm just counting the lines needed to get Squiggle and SquigglePy to output a model, not the lines inside them.
## Notes
@ -93,6 +97,24 @@ For the Python code, it's possible that the lack of speed is more a function of
R has a warm place in my heart from back in the day, and it has predefined functions to do everything. It was particularly fast to write for me, though not particularly fast to run :) However, I do recall that R does have some multithreading support; it wasn't used.
### Lua
I was also really pleased with Lua. I liked the simplicity and elegance, and lack of warts. And coming from javascript, I appreciated that the program did not fail silently, but rather gave me useful errors, like:
```
lua samples.lua
lua: samples.lua:42: table index is nil
stack traceback:
samples.lua:42: in function 'mixture'
samples.lua:49: in main chunk
[C]: in ?
make: *** [makefile:14: run] Error 1
```
I also appreciated the speedup when using the LuaJIT interpreter.
Overall I'm thinking that a combination of lua at least for scripting and ¿nim/C/tbd?
### Overall thoughts
Overall I don't think that this is a fair comparison of the languages intrinsically, because I'm just differentially good at them, because I've chosen to put more effort in ones than in others. But it is still useful to me personally, and perhaps mildly informative to others.
@ -105,6 +127,7 @@ Overall I don't think that this is a fair comparison of the languages intrinsica
- [ ] Go
- [ ] Zig
- [ ] Forth
- [ ] sh/bash, lol?
- [ ] OCaml
- [ ] Haskell
- [ ] CUDA
@ -112,6 +135,17 @@ Overall I don't think that this is a fair comparison of the languages intrinsica
- [ ] Maybe still worth reversing the process?
- ... and suggestions welcome
## Languages added so far
- [x] Squiggle
- [x] Javascript (NodeJS)
- [x] Python (CPython)
- [x] R
- [x] C
- [x] Nim
- [x] SquigglePy
- [x] Lua
## Roadmap
The future of this project is uncertain. In most words, I simply forget about this repository.

View File

@ -9,22 +9,24 @@
# make profile-linux
SRC=samples.lua
# INTERPETER=lua# < original
INTERPETER=luajit# < faster
run: $(SRC)
lua $(SRC)
$(INTERPETER) $(SRC)
time-linux:
@echo "Requires /bin/time, found on GNU/Linux systems" && echo
@echo "Running 100x and taking avg time of: lua $(SRC)"
@t=$$(/usr/bin/time -f "%e" -p bash -c 'for i in {1..100}; do lua $(SRC); done' 2>&1 >/dev/null | grep real | awk '{print $$2}' ); echo "scale=2; 1000 * $$t / 100" | bc | sed "s|^|Time: |" | sed 's|$$|ms|' && echo
@echo "Running 100x and taking avg time of: $(INTERPETER) $(SRC)"
@t=$$(/usr/bin/time -f "%e" -p bash -c 'for i in {1..100}; do $(INTERPETER) $(SRC); done' 2>&1 >/dev/null | grep real | awk '{print $$2}' ); echo "scale=2; 1000 * $$t / 100" | bc | sed "s|^|Time: |" | sed 's|$$|ms|' && echo
time-linux-simple:
@echo "Requires /bin/time, found on GNU/Linux systems" && echo
/bin/time -f "Time: %es" lua $(SRC) && echo
/bin/time -f "Time: %es" $(INTERPETER) $(SRC) && echo
profile-linux:
@echo "Requires perf, which depends on the kernel version, and might be in linux-tools package or similar"
@echo "Must be run as sudo"
sudo perf record lua $(SRC)
sudo perf record $(INTERPETER) $(SRC)
sudo perf report
rm perf.data

View File

@ -11,3 +11,8 @@ examine: samples
nim c $(VERBOSE) samples.nim && time ./samples $(VERBOSE) && echo
nim c $(VERBOSE) -d:release samples.nim && time ./samples $(VERBOSE) && echo
nim c $(VERBOSE) -d:danger samples.nim && time ./samples $(VERBOSE)
time-linux:
@echo "Requires /bin/time, found on GNU/Linux systems" && echo
@echo "Running 100x and taking avg time of: $(OUTPUT)"
@t=$$(/usr/bin/time -f "%e" -p bash -c 'for i in {1..100}; do ./samples; done' 2>&1 >/dev/null | grep real | awk '{print $$2}' ); echo "scale=2; 1000 * $$t / 100" | bc | sed "s|^|Time: |" | sed 's|$$|ms|' && echo

View File

@ -89,6 +89,12 @@ real 0m0.068s
user 0m0.048s
sys 0m0.020s
— make time-linux
Requires /bin/time, found on GNU/Linux systems
Running 100x and taking avg time of:
Time: 38.90ms
## Squigglepy
@ -101,3 +107,24 @@ python3 samples.py
real 0m1.602s
user 0m1.582s
sys 0m0.331s
## Lua
— make time-linux-simple
Requires /bin/time, found on GNU/Linux systems
/bin/time -f "Time: %es" lua samples.lua && echo
0.88932198462254
Time: 0.28s
— make time-linux
Requires /bin/time, found on GNU/Linux systems
Running 100x and taking avg time of: lua samples.lua
Time: 275.80ms
— make time-linux
Requires /bin/time, found on GNU/Linux systems
Running 100x and taking avg time of: luajit samples.lua
Time: 68.60ms