Merge remote-tracking branch 'origin/develop' into log-score-attempt

This commit is contained in:
Quinn Dougherty 2022-05-02 11:34:29 -04:00
commit 58c885f963
36 changed files with 1720 additions and 876 deletions

View File

@ -1,12 +1,22 @@
# Squiggle
[![Packages check](https://github.com/quantified-uncertainty/squiggle/actions/workflows/ci.yml/badge.svg)](https://github.com/quantified-uncertainty/squiggle/actions/workflows/ci.yml)
[![npm version](https://badge.fury.io/js/@quri%2Fsquiggle-lang.svg)](https://www.npmjs.com/package/@quri/squiggle-lang)
[![npm version](https://badge.fury.io/js/@quri%2Fsquiggle-components.svg)](https://www.npmjs.com/package/@quri/squiggle-components)
[![npm version - lang](https://badge.fury.io/js/@quri%2Fsquiggle-lang.svg)](https://www.npmjs.com/package/@quri/squiggle-lang)
[![npm version - components](https://badge.fury.io/js/@quri%2Fsquiggle-components.svg)](https://www.npmjs.com/package/@quri/squiggle-components)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/quantified-uncertainty/squiggle/blob/develop/LICENSE)
[![codecov](https://codecov.io/gh/quantified-uncertainty/squiggle/branch/develop/graph/badge.svg?token=QRLBL5CQ7C)](https://codecov.io/gh/quantified-uncertainty/squiggle)
This is an experimental DSL/language for making probabilistic estimates. The full story can be found [here](https://www.lesswrong.com/s/rDe8QE5NvXcZYzgZ3).
_An estimation language_.
## Get started
- [Gallery](https://www.squiggle-language.com/docs/Discussions/Gallery)
- [Squiggle playground](https://squiggle-language.com/playground)
- [Language basics](https://www.squiggle-language.com/docs/Features/Language)
- [Squiggle functions source of truth](https://www.squiggle-language.com/docs/Features/Functions)
- [Known bugs](https://www.squiggle-language.com/docs/Discussions/Bugs)
- [Original lesswrong sequence](https://www.lesswrong.com/s/rDe8QE5NvXcZYzgZ3)
- [Author your squiggle models as Observable notebooks](https://observablehq.com/@hazelfire/squiggle)
## Our deployments
@ -27,7 +37,7 @@ the packages can be found in `packages`.
- `@quri/squiggle-components` in `packages/components` contains React components that
can be passed squiggle strings as props, and return a presentation of the result
of the calculation.
- `@quri/squiggle-website` in `packages/website` The main descriptive website for squiggle,
- `packages/website` is the main descriptive website for squiggle,
it is hosted at `squiggle-language.com`.
The playground depends on the components library which then depends on the language. This means that if you wish to work on the components library, you will need to build (no need to bundle) the language, and as of this writing playground doesn't really work.

View File

@ -1,8 +1,29 @@
# Squiggle Components
[![npm version](https://badge.fury.io/js/@quri%2Fsquiggle-components.svg)](https://www.npmjs.com/package/@quri/squiggle-components)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/quantified-uncertainty/squiggle/blob/develop/LICENSE)
This package contains all the components for squiggle. These can be used either as a library or hosted as a [storybook](https://storybook.js.org/).
# Squiggle components
# Build for development
This package contains the react components for squiggle. These can be used either as a library or hosted as a [storybook](https://storybook.js.org/).
# Usage in a `react` project
For example, in a fresh `create-react-app` project
```sh
yarn add @quri/squiggle-components
```
Add to `App.js`:
```jsx
import { SquiggleEditor } from "@quri/squiggle-components";
<SquiggleEditor
initialSquiggleString="x = beta($alpha, 10); x + $shift"
jsImports={{ alpha: 3, shift: 20 }}
/>;
```
# Build storybook for development
We assume that you had run `yarn` at monorepo level, installing dependencies.
@ -20,10 +41,3 @@ Run a development server
```sh
yarn start
```
And build artefacts for production,
```sh
yarn bundle # builds components library
yarn build # builds storybook app
```

View File

@ -1,13 +1,19 @@
{
"name": "@quri/squiggle-components",
"version": "0.2.9",
"licence": "MIT",
"version": "0.2.17",
"license": "MIT",
"dependencies": {
"antd": "^4.20.1",
"react-ace": "10.1.0",
"react-dom": "^18.1.0",
"@quri/squiggle-lang": "^0.2.8",
"@react-hook/size": "^2.1.2",
"styled-components": "^5.3.5"
"lodash": "^4.17.21",
"react": "^18.1.0",
"react-ace": "^10.1.0",
"react-dom": "^18.1.0",
"react-vega": "^7.5.0",
"styled-components": "^5.3.5",
"vega": "^5.22.1",
"vega-embed": "^6.20.6",
"vega-lite": "^5.2.0"
},
"devDependencies": {
"@babel/plugin-proposal-private-property-in-object": "^7.16.7",
@ -19,34 +25,26 @@
"@storybook/node-logger": "^6.4.22",
"@storybook/preset-create-react-app": "^4.1.0",
"@storybook/react": "^6.4.22",
"@types/styled-components": "^5.1.24",
"@types/webpack": "^5.28.0",
"style-loader": "^3.3.1",
"ts-loader": "^9.2.9",
"webpack": "^5.72.0",
"webpack-cli": "^4.9.2",
"webpack-dev-server": "^4.8.1",
"@quri/squiggle-lang": "0.2.5",
"@testing-library/jest-dom": "^5.16.4",
"@testing-library/react": "^13.1.1",
"@testing-library/user-event": "^14.1.1",
"@types/jest": "^27.4.0",
"@types/lodash": "^4.14.182",
"@types/node": "^17.0.29",
"@types/node": "^17.0.31",
"@types/react": "^18.0.3",
"@types/react-dom": "^18.0.2",
"@types/styled-components": "^5.1.24",
"@types/webpack": "^5.28.0",
"cross-env": "^7.0.3",
"lodash": "^4.17.21",
"react": "^18.1.0",
"react-scripts": "5.0.1",
"react-vega": "^7.5.0",
"react-scripts": "^5.0.1",
"style-loader": "^3.3.1",
"ts-loader": "^9.3.0",
"tsconfig-paths-webpack-plugin": "^3.5.2",
"typescript": "^4.6.3",
"vega": "^5.22.1",
"vega-embed": "^6.20.6",
"vega-lite": "^5.2.0",
"web-vitals": "^2.1.4",
"webpack-cli": "^4.9.2"
"webpack": "^5.72.0",
"webpack-cli": "^4.9.2",
"webpack-dev-server": "^4.8.1"
},
"scripts": {
"start": "cross-env REACT_APP_FAST_REFRESH=false && start-storybook -p 6006 -s public",
@ -54,7 +52,8 @@
"bundle": "webpack",
"all": "yarn bundle && yarn build",
"lint": "prettier --check .",
"format": "prettier --write ."
"format": "prettier --write .",
"prepack": "yarn bundle && tsc -b"
},
"eslintConfig": {
"extends": [
@ -88,6 +87,6 @@
"@types/react": "17.0.43"
},
"source": "./src/index.ts",
"main": "dist/bundle.js",
"types": "dist/src/index.d.ts"
"main": "./dist/src/index.js",
"types": "./dist/src/index.d.ts"
}

View File

@ -6,6 +6,7 @@ import { distributionErrorToString } from "@quri/squiggle-lang";
import { createClassFromSpec } from "react-vega";
import * as chartSpecification from "../vega-specs/spec-distributions.json";
import { ErrorBox } from "./ErrorBox";
import styled from "styled-components";
let SquiggleVegaChart = createClassFromSpec({
spec: chartSpecification as Spec,
@ -24,19 +25,21 @@ export const DistributionChart: React.FC<DistributionChartProps> = ({
}: DistributionChartProps) => {
let shape = distribution.pointSet();
if (shape.tag === "Ok") {
return (
let widthProp = width ? width - 20 : undefined;
var result = (
<SquiggleVegaChart
data={{ con: shape.value.continuous, dis: shape.value.discrete }}
width={width - 20}
width={widthProp}
height={height}
actions={false}
/>
);
} else {
return (
var result = (
<ErrorBox heading="Distribution Error">
{distributionErrorToString(shape.value)}
</ErrorBox>
);
}
return result;
};

View File

@ -5,12 +5,15 @@ import {
run,
errorValueToString,
squiggleExpression,
bindings,
samplingParams,
jsImports,
defaultImports,
defaultBindings,
} from "@quri/squiggle-lang";
import type { samplingParams } from "@quri/squiggle-lang";
import { NumberShower } from "./NumberShower";
import { DistributionChart } from "./DistributionChart";
import { ErrorBox } from "./ErrorBox";
import useSize from "@react-hook/size";
const variableBox = {
Component: styled.div`
@ -44,6 +47,8 @@ export const VariableBox: React.FC<{
);
};
let RecordKeyHeader = styled.h3``;
export interface SquiggleItemProps {
/** The input string for squiggle */
expression: squiggleExpression;
@ -104,6 +109,17 @@ const SquiggleItem: React.FC<SquiggleItemProps> = ({
))}
</VariableBox>
);
case "record":
return (
<VariableBox heading="Record">
{Object.entries(expression.value).map(([key, r]) => (
<>
<RecordKeyHeader>{key}</RecordKeyHeader>
<SquiggleItem expression={r} width={width - 20} height={50} />
</>
))}
</VariableBox>
);
default:
return (
<ErrorBox heading="No Viewer">
@ -128,47 +144,56 @@ export interface SquiggleChartProps {
diagramStop?: number;
/** If the result is a function, how many points along the function it samples */
diagramCount?: number;
/** variables declared before this expression */
environment?: unknown;
/** When the environment changes */
onChange?(expr: squiggleExpression): void;
/** CSS width of the element */
width?: number;
height?: number;
/** Bindings of previous variables declared */
bindings?: bindings;
/** JS imported parameters */
jsImports?: jsImports;
}
const ChartWrapper = styled.div`
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto,
"Helvetica Neue", Arial, "Noto Sans", sans-serif, "Apple Color Emoji",
"Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji";
`;
export const SquiggleChart: React.FC<SquiggleChartProps> = ({
squiggleString = "",
sampleCount = 1000,
outputXYPoints = 1000,
onChange = () => {},
height = 60,
bindings = defaultBindings,
jsImports = defaultImports,
width = NaN,
}: SquiggleChartProps) => {
const target = React.useRef(null);
const [componentWidth] = useSize(target);
// I would have wanted to just use componentWidth, but this created infinite loops with SquiggleChart.stories.
//So you can manually add a width, as an escape hatch.
let _width = width || componentWidth;
let samplingInputs: samplingParams = {
sampleCount: sampleCount,
xyPointLength: outputXYPoints,
};
let expressionResult = run(squiggleString, samplingInputs);
let expressionResult = run(
squiggleString,
bindings,
samplingInputs,
jsImports
);
let internal: JSX.Element;
if (expressionResult.tag === "Ok") {
let expression = expressionResult.value;
onChange(expression);
internal = (
<SquiggleItem expression={expression} width={_width} height={height} />
<SquiggleItem expression={expression} width={width} height={height} />
);
} else {
// At this point, we came across an error. What was our error?
internal = (
<ErrorBox heading={"Parse Error"}>
{errorValueToString(expressionResult.value)}
</ErrorBox>
);
}
return <div ref={target}>{internal}</div>;
return <ChartWrapper>{internal}</ChartWrapper>;
};

View File

@ -3,7 +3,19 @@ import * as ReactDOM from "react-dom";
import { SquiggleChart } from "./SquiggleChart";
import { CodeEditor } from "./CodeEditor";
import styled from "styled-components";
import type { squiggleExpression } from "@quri/squiggle-lang";
import type {
squiggleExpression,
samplingParams,
bindings,
jsImports,
} from "@quri/squiggle-lang";
import {
runPartial,
errorValueToString,
defaultImports,
defaultBindings,
} from "@quri/squiggle-lang";
import { ErrorBox } from "./ErrorBox";
export interface SquiggleEditorProps {
/** The input string for squiggle */
@ -20,12 +32,14 @@ export interface SquiggleEditorProps {
diagramStop?: number;
/** If the result is a function, how many points along the function it samples */
diagramCount?: number;
/** The environment, other variables that were already declared */
environment?: unknown;
/** when the environment changes. Used again for notebook magic*/
onChange?(expr: squiggleExpression): void;
/** The width of the element */
width: number;
width?: number;
/** Previous variable declarations */
bindings?: bindings;
/** JS Imports */
jsImports?: jsImports;
}
const Input = styled.div`
@ -45,7 +59,8 @@ export let SquiggleEditor: React.FC<SquiggleEditorProps> = ({
diagramStop,
diagramCount,
onChange,
environment,
bindings = defaultBindings,
jsImports = defaultImports,
}: SquiggleEditorProps) => {
let [expression, setExpression] = React.useState(initialSquiggleString);
return (
@ -69,8 +84,9 @@ export let SquiggleEditor: React.FC<SquiggleEditorProps> = ({
diagramStart={diagramStart}
diagramStop={diagramStop}
diagramCount={diagramCount}
environment={environment}
onChange={onChange}
bindings={bindings}
jsImports={jsImports}
/>
</div>
);
@ -107,3 +123,88 @@ export function renderSquiggleEditorToDom(props: SquiggleEditorProps) {
);
return parent;
}
export interface SquigglePartialProps {
/** The input string for squiggle */
initialSquiggleString?: string;
/** If the output requires monte carlo sampling, the amount of samples */
sampleCount?: number;
/** The amount of points returned to draw the distribution */
outputXYPoints?: number;
kernelWidth?: number;
pointDistLength?: number;
/** If the result is a function, where the function starts */
diagramStart?: number;
/** If the result is a function, where the function ends */
diagramStop?: number;
/** If the result is a function, how many points along the function it samples */
diagramCount?: number;
/** when the environment changes. Used again for notebook magic*/
onChange?(expr: bindings): void;
/** Previously declared variables */
bindings?: bindings;
/** Variables imported from js */
jsImports?: jsImports;
}
export let SquigglePartial: React.FC<SquigglePartialProps> = ({
initialSquiggleString = "",
onChange,
bindings = defaultBindings,
sampleCount = 1000,
outputXYPoints = 1000,
jsImports = defaultImports,
}: SquigglePartialProps) => {
let samplingInputs: samplingParams = {
sampleCount: sampleCount,
xyPointLength: outputXYPoints,
};
let [expression, setExpression] = React.useState(initialSquiggleString);
let squiggleResult = runPartial(
expression,
bindings,
samplingInputs,
jsImports
);
if (squiggleResult.tag == "Ok") {
if (onChange) onChange(squiggleResult.value);
}
return (
<div>
<Input>
<CodeEditor
value={expression}
onChange={setExpression}
oneLine={true}
showGutter={false}
height={20}
/>
</Input>
{squiggleResult.tag == "Error" ? (
<ErrorBox heading="Error">
{errorValueToString(squiggleResult.value)}
</ErrorBox>
) : (
<></>
)}
</div>
);
};
export function renderSquigglePartialToDom(props: SquigglePartialProps) {
let parent = document.createElement("div");
ReactDOM.render(
<SquigglePartial
{...props}
onChange={(bindings) => {
// @ts-ignore
parent.value = bindings;
parent.dispatchEvent(new CustomEvent("input"));
if (props.onChange) props.onChange(bindings);
}}
/>,
parent
);
return parent;
}

View File

@ -1,11 +1,9 @@
import _ from "lodash";
import React, { FC, useState } from "react";
import React, { FC, ReactElement, useState } from "react";
import ReactDOM from "react-dom";
import { SquiggleChart } from "./SquiggleChart";
import CodeEditor from "./CodeEditor";
import { Form, Input, Row, Col } from "antd";
import styled from "styled-components";
import "antd/dist/antd.css";
interface FieldFloatProps {
label: string;
@ -14,10 +12,19 @@ interface FieldFloatProps {
onChange: (value: number) => void;
}
const Input = styled.input``;
const FormItem = (props: { label: string; children: ReactElement }) => (
<div>
<label>{props.label}</label>
{props.children}
</div>
);
function FieldFloat(Props: FieldFloatProps) {
let [contents, setContents] = useState(Props.value + "");
return (
<Form.Item label={Props.label}>
<FormItem label={Props.label}>
<Input
value={contents}
className={Props.className ? Props.className : ""}
@ -29,7 +36,7 @@ function FieldFloat(Props: FieldFloatProps) {
}
}}
/>
</Form.Item>
</FormItem>
);
}
@ -65,6 +72,12 @@ const Display = styled.div<TitleProps>`
max-height: ${(props) => props.maxHeight}px;
`;
const Row = styled.div`
display: grid;
grid-template-columns: 1fr 1fr;
`;
const Col = styled.div``;
let SquigglePlayground: FC<Props> = ({
initialSquiggleString = "",
height = 300,
@ -79,7 +92,7 @@ let SquigglePlayground: FC<Props> = ({
return (
<ShowBox height={height}>
<Row>
<Col span={12}>
<Col>
<CodeEditor
value={squiggleString}
onChange={setSquiggleString}
@ -88,7 +101,7 @@ let SquigglePlayground: FC<Props> = ({
height={height - 3}
/>
</Col>
<Col span={12}>
<Col>
<Display maxHeight={height - 3}>
<SquiggleChart
squiggleString={squiggleString}

View File

@ -1,7 +1,9 @@
export { SquiggleChart } from "./components/SquiggleChart";
export {
SquiggleEditor,
SquigglePartial,
renderSquiggleEditorToDom,
renderSquigglePartialToDom,
} from "./components/SquiggleEditor";
import SquigglePlayground, {
renderSquigglePlaygroundToDom,

View File

@ -3,6 +3,7 @@
"description": "A basic area chart example",
"width": 500,
"height": 100,
"autosize": "fit",
"padding": 5,
"data": [
{
@ -87,7 +88,7 @@
"tickOpacity": 0.0,
"domainColor": "#fff",
"domainOpacity": 0.0,
"format": "~s",
"format": "~g",
"tickCount": 10
}
],

View File

@ -2,7 +2,6 @@ node_modules
shell.nix
.cache
.direnv
src
__tests__
lib
examples

View File

@ -1,6 +1,28 @@
[![npm version](https://badge.fury.io/js/@quri%2Fsquiggle-lang.svg)](https://www.npmjs.com/package/@quri/squiggle-lang)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/quantified-uncertainty/squiggle/blob/develop/LICENSE)
# Squiggle language
## Build for development
_An estimation language_
# Use the `npm` package
For instance, in a javascript project, you can
```sh
yarn add @quri/squiggle-lang
```
```js
import { run } from "@quri/squiggle-lang";
run(
"normal(0, 1) * fromSamples([-3,-2,-1,1,2,3,3,3,4,9]"
).value.value.toSparkline().value;
```
**However, for most use cases you'll prefer to use our [library of react components](https://www.npmjs.com/package/@quri/squiggle-components)**, and let your app transitively depend on `@quri/squiggle-lang`.
# Build for development
We assume that you ran `yarn` at the monorepo level.
@ -15,13 +37,16 @@ Other:
```sh
yarn start # listens to files and recompiles at every mutation
yarn test
yarn test:watch # keeps an active session and runs all tests at every mutation
# where o := open in osx and o := xdg-open in linux,
yarn coverage; o _coverage/index.html # produces coverage report and opens it in browser
yarn coverage:rescript; o _coverage/index.html # produces coverage report and opens it in browser
```
## Information
# Distributing this package or using this package from other monorepo packages
As it says in the other `packages/*/README.md`s, building this package is an essential step of building other packages.
# Information
Squiggle is a language for representing probability distributions, as well as functions that return probability distributions. Its original intended use is for improving epistemics around EA decisions.
@ -34,11 +59,3 @@ This package is mainly written in [ReScript](https://rescript-lang.org/), but ha
ReScript has an interesting philosophy of not providing much in the way of effective build tools. Every ReScript file is compiled into `.bs.js` and `.gen.ts` files with the same name and same location, and then you can use these files in other `.js` files to create your program. To generate these files to build the package, you run `yarn build`.
`.gen.ts` files are created by the [`@genType`](https://rescript-lang.org/docs/gentype/latest/getting-started) decorator, which creates typescript typings for needed parts of the codebase so that they can be easily used in typescript. These .gen.ts files reference the .bs.js files generated by rescript.
### Errors regarding the `rationale` package
You may notice sometimes, that there are errors about the `rationale` package. If you ever get these errors, `yarn build` should fix this issue. These errors occur because `yarn build` also needs to create build files that are in `node_modules`. So if you replace `node_modules` you may need to rebuild to get those files back.
## Distributing this package or using this package from other monorepo packages
As it says in the other `packages/*/README.md`s, building this package is an essential step of building other packages.

View File

@ -1,23 +1,5 @@
import {
run,
Distribution,
resultMap,
squiggleExpression,
errorValueToString,
} from "../../src/js/index";
let testRun = (x: string): squiggleExpression => {
let result = run(x, { sampleCount: 100, xyPointLength: 100 });
expect(result.tag).toEqual("Ok");
if (result.tag === "Ok") {
return result.value;
} else {
throw Error(
"Expected squiggle expression to evaluate but got error: " +
errorValueToString(result.value)
);
}
};
import { Distribution, resultMap, defaultBindings } from "../../src/js/index";
import { testRun, testRunPartial } from "./TestHelpers";
function Ok<b>(x: b) {
return { tag: "Ok", value: x };
@ -42,6 +24,72 @@ describe("Log function", () => {
});
});
describe("Array", () => {
test("nested Array", () => {
expect(testRun("[[1]]")).toEqual({
tag: "array",
value: [
{
tag: "array",
value: [
{
tag: "number",
value: 1,
},
],
},
],
});
});
});
describe("Record", () => {
test("Return record", () => {
expect(testRun("{a: 1}")).toEqual({
tag: "record",
value: {
a: {
tag: "number",
value: 1,
},
},
});
});
});
describe("Partials", () => {
test("Can pass variables between partials and cells", () => {
let bindings = testRunPartial(`x = 5`);
let bindings2 = testRunPartial(`y = x + 2`, bindings);
expect(testRun(`y + 3`, bindings2)).toEqual({
tag: "number",
value: 10,
});
});
});
describe("JS Imports", () => {
test("Can pass parameters into partials and cells", () => {
let bindings = testRunPartial(`y = $x + 2`, defaultBindings, { x: 1 });
let bindings2 = testRunPartial(`z = y + $a`, bindings, { a: 3 });
expect(testRun(`z`, bindings2)).toEqual({
tag: "number",
value: 6,
});
});
test("Complicated deep parameters", () => {
expect(
testRun(`$x.y[0][0].w + $x.z + $u.v`, defaultBindings, {
x: { y: [[{ w: 1 }]], z: 2 },
u: { v: 3 },
})
).toEqual({
tag: "number",
value: 6,
});
});
});
describe("Distribution", () => {
//It's important that sampleCount is less than 9. If it's more, than that will create randomness
//Also, note, the value should be created using makeSampleSetDist() later on.

View File

@ -1,5 +1,5 @@
import { Distribution } from "../../src/js/index";
import { expectErrorToBeBounded, failDefault } from "./TestHelpers";
import { expectErrorToBeBounded, failDefault, testRun } from "./TestHelpers";
import * as fc from "fast-check";
// Beware: float64Array makes it appear in an infinite loop.
@ -212,3 +212,18 @@ describe("mean is mean", () => {
);
});
});
describe("fromSamples function", () => {
test.skip("gives a mean near the mean of the input", () => {
fc.assert(
fc.property(arrayGen(), (xs_) => {
let xs = Array.from(xs_);
let xsString = xs.toString();
let squiggleString = `x = fromSamples([${xsString}]); mean(x)`;
let squiggleResult = testRun(squiggleString);
let mean = xs.reduce((a, b) => a + b, 0.0) / xs.length;
expect(squiggleResult.value).toBeCloseTo(mean, 4);
})
);
});
});

View File

@ -1,15 +1,53 @@
import {
run,
// Distribution,
runPartial,
bindings,
squiggleExpression,
errorValueToString,
// errorValue,
// result,
defaultImports,
defaultBindings,
jsImports,
} from "../../src/js/index";
export function testRun(x: string): squiggleExpression {
let squiggleResult = run(x, { sampleCount: 1000, xyPointLength: 100 });
// return squiggleResult.value
export function testRun(
x: string,
bindings: bindings = defaultBindings,
imports: jsImports = defaultImports
): squiggleExpression {
let squiggleResult = run(
x,
bindings,
{
sampleCount: 1000,
xyPointLength: 100,
},
imports
);
if (squiggleResult.tag === "Ok") {
return squiggleResult.value;
} else {
throw new Error(
`Expected squiggle expression to evaluate but got error: ${errorValueToString(
squiggleResult.value
)}`
);
}
}
export function testRunPartial(
x: string,
bindings: bindings = defaultBindings,
imports: jsImports = defaultImports
): bindings {
let squiggleResult = runPartial(
x,
bindings,
{
sampleCount: 1000,
xyPointLength: 100,
},
imports
);
if (squiggleResult.tag === "Ok") {
return squiggleResult.value;
} else {

View File

@ -1,13 +1,13 @@
{
"name": "@quri/squiggle-lang",
"version": "0.2.5",
"version": "0.2.8",
"homepage": "https://squiggle-language.com",
"licence": "MIT",
"license": "MIT",
"scripts": {
"build": "rescript build -with-deps",
"build": "rescript build -with-deps && tsc",
"bundle": "webpack",
"start": "rescript build -w -with-deps",
"clean": "rescript clean",
"clean": "rescript clean && rm -r dist",
"test:reducer": "jest __tests__/Reducer*/",
"benchmark": "ts-node benchmark/conversion_tests.ts",
"test": "jest",
@ -24,6 +24,7 @@
"format:rescript": "rescript format -all",
"format:prettier": "prettier --write .",
"format": "yarn format:rescript && yarn format:prettier",
"prepack": "yarn build && yarn test && yarn bundle",
"all": "yarn build && yarn bundle && yarn test"
},
"keywords": [
@ -31,34 +32,36 @@
],
"author": "Quantified Uncertainty Research Institute",
"license": "MIT",
"dependencies": {
"rescript": "^9.1.4",
"jstat": "^1.9.5",
"pdfast": "^0.2.0",
"mathjs": "^10.5.0"
},
"devDependencies": {
"bisect_ppx": "^2.7.1",
"jstat": "^1.9.5",
"lodash": "4.17.21",
"rescript": "^9.1.4",
"lodash": "^4.17.21",
"rescript-fast-check": "^1.1.1",
"@glennsl/rescript-jest": "^0.9.0",
"@istanbuljs/nyc-config-typescript": "^1.0.2",
"@types/jest": "^27.4.0",
"babel-plugin-transform-es2015-modules-commonjs": "^6.26.2",
"chalk": "^5.0.1",
"codecov": "3.8.3",
"fast-check": "2.25.0",
"codecov": "^3.8.3",
"fast-check": "^2.25.0",
"gentype": "^4.3.0",
"jest": "^27.5.1",
"mathjs": "10.5.0",
"moduleserve": "0.9.1",
"moduleserve": "^0.9.1",
"nyc": "^15.1.0",
"pdfast": "^0.2.0",
"reanalyze": "^2.19.0",
"ts-jest": "^27.1.4",
"ts-loader": "^9.2.8",
"ts-loader": "^9.3.0",
"ts-node": "^10.7.0",
"typescript": "^4.6.3",
"webpack": "^5.72.0",
"webpack-cli": "^4.9.2"
},
"source": "./src/js/index.ts",
"main": "./dist/bundle.js",
"types": "./dist/js/index.d.ts"
"main": "./dist/src/js/index.js",
"types": "./dist/src/js/index.d.ts"
}

View File

@ -0,0 +1,247 @@
import * as _ from "lodash";
import {
genericDist,
continuousShape,
discreteShape,
samplingParams,
distributionError,
toPointSet,
distributionErrorToString,
} from "../rescript/TypescriptInterface.gen";
import { result, resultMap, Ok } from "./types";
import {
Constructors_mean,
Constructors_sample,
Constructors_pdf,
Constructors_cdf,
Constructors_inv,
Constructors_normalize,
Constructors_isNormalized,
Constructors_toPointSet,
Constructors_toSampleSet,
Constructors_truncate,
Constructors_inspect,
Constructors_toString,
Constructors_toSparkline,
Constructors_algebraicAdd,
Constructors_algebraicMultiply,
Constructors_algebraicDivide,
Constructors_algebraicSubtract,
Constructors_algebraicLogarithm,
Constructors_algebraicPower,
Constructors_pointwiseAdd,
Constructors_pointwiseMultiply,
Constructors_pointwiseDivide,
Constructors_pointwiseSubtract,
Constructors_pointwiseLogarithm,
Constructors_pointwisePower,
} from "../rescript/Distributions/DistributionOperation/DistributionOperation.gen";
export type point = { x: number; y: number };
function shapePoints(x: continuousShape | discreteShape): point[] {
let xs = x.xyShape.xs;
let ys = x.xyShape.ys;
return _.zipWith(xs, ys, (x, y) => ({ x, y }));
}
export type shape = {
continuous: point[];
discrete: point[];
};
export class Distribution {
t: genericDist;
env: samplingParams;
constructor(t: genericDist, env: samplingParams) {
this.t = t;
this.env = env;
return this;
}
mapResultDist(
r: result<genericDist, distributionError>
): result<Distribution, distributionError> {
return resultMap(r, (v: genericDist) => new Distribution(v, this.env));
}
mean(): result<number, distributionError> {
return Constructors_mean({ env: this.env }, this.t);
}
sample(): result<number, distributionError> {
return Constructors_sample({ env: this.env }, this.t);
}
pdf(n: number): result<number, distributionError> {
return Constructors_pdf({ env: this.env }, this.t, n);
}
cdf(n: number): result<number, distributionError> {
return Constructors_cdf({ env: this.env }, this.t, n);
}
inv(n: number): result<number, distributionError> {
return Constructors_inv({ env: this.env }, this.t, n);
}
isNormalized(): result<boolean, distributionError> {
return Constructors_isNormalized({ env: this.env }, this.t);
}
normalize(): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_normalize({ env: this.env }, this.t)
);
}
type() {
return this.t.tag;
}
pointSet(): result<shape, distributionError> {
let pointSet = toPointSet(
this.t,
{
xyPointLength: this.env.xyPointLength,
sampleCount: this.env.sampleCount,
},
undefined
);
if (pointSet.tag === "Ok") {
let distribution = pointSet.value;
if (distribution.tag === "Continuous") {
return Ok({
continuous: shapePoints(distribution.value),
discrete: [],
});
} else if (distribution.tag === "Discrete") {
return Ok({
discrete: shapePoints(distribution.value),
continuous: [],
});
} else {
return Ok({
discrete: shapePoints(distribution.value.discrete),
continuous: shapePoints(distribution.value.continuous),
});
}
} else {
return pointSet;
}
}
toPointSet(): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_toPointSet({ env: this.env }, this.t)
);
}
toSampleSet(n: number): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_toSampleSet({ env: this.env }, this.t, n)
);
}
truncate(
left: number,
right: number
): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_truncate({ env: this.env }, this.t, left, right)
);
}
inspect(): result<Distribution, distributionError> {
return this.mapResultDist(Constructors_inspect({ env: this.env }, this.t));
}
toString(): string {
let result = Constructors_toString({ env: this.env }, this.t);
if (result.tag === "Ok") {
return result.value;
} else {
return distributionErrorToString(result.value);
}
}
toSparkline(n: number): result<string, distributionError> {
return Constructors_toSparkline({ env: this.env }, this.t, n);
}
algebraicAdd(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_algebraicAdd({ env: this.env }, this.t, d2.t)
);
}
algebraicMultiply(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_algebraicMultiply({ env: this.env }, this.t, d2.t)
);
}
algebraicDivide(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_algebraicDivide({ env: this.env }, this.t, d2.t)
);
}
algebraicSubtract(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_algebraicSubtract({ env: this.env }, this.t, d2.t)
);
}
algebraicLogarithm(
d2: Distribution
): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_algebraicLogarithm({ env: this.env }, this.t, d2.t)
);
}
algebraicPower(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_algebraicPower({ env: this.env }, this.t, d2.t)
);
}
pointwiseAdd(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_pointwiseAdd({ env: this.env }, this.t, d2.t)
);
}
pointwiseMultiply(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_pointwiseMultiply({ env: this.env }, this.t, d2.t)
);
}
pointwiseDivide(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_pointwiseDivide({ env: this.env }, this.t, d2.t)
);
}
pointwiseSubtract(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_pointwiseSubtract({ env: this.env }, this.t, d2.t)
);
}
pointwiseLogarithm(
d2: Distribution
): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_pointwiseLogarithm({ env: this.env }, this.t, d2.t)
);
}
pointwisePower(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_pointwisePower({ env: this.env }, this.t, d2.t)
);
}
}

View File

@ -1,115 +1,127 @@
import * as _ from "lodash";
import {
genericDist,
samplingParams,
evaluate,
evaluateUsingExternalBindings,
evaluatePartialUsingExternalBindings,
externalBindings,
expressionValue,
errorValue,
distributionError,
toPointSet,
continuousShape,
discreteShape,
distributionErrorToString,
} from "../rescript/TypescriptInterface.gen";
export {
makeSampleSetDist,
errorValueToString,
distributionErrorToString,
} from "../rescript/TypescriptInterface.gen";
export type {
samplingParams,
errorValue,
externalBindings as bindings,
jsImports,
};
import {
Constructors_mean,
Constructors_sample,
Constructors_pdf,
Constructors_cdf,
Constructors_inv,
Constructors_normalize,
Constructors_isNormalized,
Constructors_toPointSet,
Constructors_toSampleSet,
Constructors_truncate,
Constructors_inspect,
Constructors_toString,
Constructors_toSparkline,
Constructors_algebraicAdd,
Constructors_algebraicMultiply,
Constructors_algebraicDivide,
Constructors_algebraicSubtract,
Constructors_algebraicLogarithm,
Constructors_algebraicPower,
Constructors_pointwiseAdd,
Constructors_pointwiseMultiply,
Constructors_pointwiseDivide,
Constructors_pointwiseSubtract,
Constructors_pointwiseLogarithm,
Constructors_pointwisePower,
} from "../rescript/Distributions/DistributionOperation/DistributionOperation.gen";
export type { samplingParams, errorValue };
jsValueToBinding,
jsValue,
rescriptExport,
squiggleExpression,
convertRawToTypescript,
} from "./rescript_interop";
import { result, resultMap, tag, tagged } from "./types";
import { Distribution } from "./distribution";
export { Distribution, squiggleExpression, result, resultMap };
export let defaultSamplingInputs: samplingParams = {
sampleCount: 10000,
xyPointLength: 10000,
};
export type result<a, b> =
| {
tag: "Ok";
value: a;
}
| {
tag: "Error";
value: b;
};
export function resultMap<a, b, c>(
r: result<a, c>,
mapFn: (x: a) => b
): result<b, c> {
if (r.tag === "Ok") {
return { tag: "Ok", value: mapFn(r.value) };
} else {
return r;
}
}
function Ok<a, b>(x: a): result<a, b> {
return { tag: "Ok", value: x };
}
type tagged<a, b> = { tag: a; value: b };
function tag<a, b>(x: a, y: b): tagged<a, b> {
return { tag: x, value: y };
}
export type squiggleExpression =
| tagged<"symbol", string>
| tagged<"string", string>
| tagged<"call", string>
| tagged<"array", squiggleExpression[]>
| tagged<"boolean", boolean>
| tagged<"distribution", Distribution>
| tagged<"number", number>
| tagged<"record", { [key: string]: squiggleExpression }>;
export function run(
squiggleString: string,
samplingInputs?: samplingParams
bindings?: externalBindings,
samplingInputs?: samplingParams,
imports?: jsImports
): result<squiggleExpression, errorValue> {
let b = bindings ? bindings : defaultBindings;
let i = imports ? imports : defaultImports;
let si: samplingParams = samplingInputs
? samplingInputs
: defaultSamplingInputs;
let result: result<expressionValue, errorValue> = evaluate(squiggleString);
let result: result<expressionValue, errorValue> =
evaluateUsingExternalBindings(squiggleString, mergeImports(b, i));
return resultMap(result, (x) => createTsExport(x, si));
}
// Run Partial. A partial is a block of code that doesn't return a value
export function runPartial(
squiggleString: string,
bindings?: externalBindings,
_samplingInputs?: samplingParams,
imports?: jsImports
): result<externalBindings, errorValue> {
let b = bindings ? bindings : defaultBindings;
let i = imports ? imports : defaultImports;
return evaluatePartialUsingExternalBindings(
squiggleString,
mergeImports(b, i)
);
}
function mergeImports(
bindings: externalBindings,
imports: jsImports
): externalBindings {
let transformedImports = Object.fromEntries(
Object.entries(imports).map(([key, value]) => [
"$" + key,
jsValueToBinding(value),
])
);
return _.merge(bindings, transformedImports);
}
type jsImports = { [key: string]: jsValue };
export let defaultImports: jsImports = {};
export let defaultBindings: externalBindings = {};
function createTsExport(
x: expressionValue,
sampEnv: samplingParams
): squiggleExpression {
switch (x.tag) {
case "EvArray":
// genType doesn't convert anything more than 2 layers down into {tag: x, value: x}
// format, leaving it as the raw values. This converts the raw values
// directly into typescript values.
//
// The casting here is because genType is about the types of the returned
// values, claiming they are fully recursive when that's not actually the
// case
return tag(
"array",
x.value.map((x) => createTsExport(x, sampEnv))
x.value.map((arrayItem): squiggleExpression => {
switch (arrayItem.tag) {
case "EvRecord":
return tag(
"record",
_.mapValues(arrayItem.value, (recordValue: unknown) =>
convertRawToTypescript(recordValue as rescriptExport, sampEnv)
)
);
case "EvArray":
let y = arrayItem.value as unknown as rescriptExport[];
return tag(
"array",
y.map((childArrayItem) =>
convertRawToTypescript(childArrayItem, sampEnv)
)
);
default:
return createTsExport(arrayItem, sampEnv);
}
})
);
case "EvBool":
return tag("boolean", x.value);
@ -120,227 +132,17 @@ function createTsExport(
case "EvNumber":
return tag("number", x.value);
case "EvRecord":
return tag(
// genType doesn't support records, so we have to do the raw conversion ourself
let result: tagged<"record", { [key: string]: squiggleExpression }> = tag(
"record",
_.mapValues(x.value, (x) => createTsExport(x, sampEnv))
_.mapValues(x.value, (x: unknown) =>
convertRawToTypescript(x as rescriptExport, sampEnv)
)
);
return result;
case "EvString":
return tag("string", x.value);
case "EvSymbol":
return tag("symbol", x.value);
}
}
export function resultExn<a, c>(r: result<a, c>): a | c {
return r.value;
}
export type point = { x: number; y: number };
export type shape = {
continuous: point[];
discrete: point[];
};
function shapePoints(x: continuousShape | discreteShape): point[] {
let xs = x.xyShape.xs;
let ys = x.xyShape.ys;
return _.zipWith(xs, ys, (x, y) => ({ x, y }));
}
export class Distribution {
t: genericDist;
env: samplingParams;
constructor(t: genericDist, env: samplingParams) {
this.t = t;
this.env = env;
return this;
}
mapResultDist(
r: result<genericDist, distributionError>
): result<Distribution, distributionError> {
return resultMap(r, (v: genericDist) => new Distribution(v, this.env));
}
mean(): result<number, distributionError> {
return Constructors_mean({ env: this.env }, this.t);
}
sample(): result<number, distributionError> {
return Constructors_sample({ env: this.env }, this.t);
}
pdf(n: number): result<number, distributionError> {
return Constructors_pdf({ env: this.env }, this.t, n);
}
cdf(n: number): result<number, distributionError> {
return Constructors_cdf({ env: this.env }, this.t, n);
}
inv(n: number): result<number, distributionError> {
return Constructors_inv({ env: this.env }, this.t, n);
}
isNormalized(): result<boolean, distributionError> {
return Constructors_isNormalized({ env: this.env }, this.t);
}
normalize(): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_normalize({ env: this.env }, this.t)
);
}
type() {
return this.t.tag;
}
pointSet(): result<shape, distributionError> {
let pointSet = toPointSet(
this.t,
{
xyPointLength: this.env.xyPointLength,
sampleCount: this.env.sampleCount,
},
undefined
);
if (pointSet.tag === "Ok") {
let distribution = pointSet.value;
if (distribution.tag === "Continuous") {
return Ok({
continuous: shapePoints(distribution.value),
discrete: [],
});
} else if (distribution.tag === "Discrete") {
return Ok({
discrete: shapePoints(distribution.value),
continuous: [],
});
} else {
return Ok({
discrete: shapePoints(distribution.value.discrete),
continuous: shapePoints(distribution.value.continuous),
});
}
} else {
return pointSet;
}
}
toPointSet(): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_toPointSet({ env: this.env }, this.t)
);
}
toSampleSet(n: number): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_toSampleSet({ env: this.env }, this.t, n)
);
}
truncate(
left: number,
right: number
): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_truncate({ env: this.env }, this.t, left, right)
);
}
inspect(): result<Distribution, distributionError> {
return this.mapResultDist(Constructors_inspect({ env: this.env }, this.t));
}
toString(): string {
let result = Constructors_toString({ env: this.env }, this.t);
if (result.tag === "Ok") {
return result.value;
} else {
return distributionErrorToString(result.value);
}
}
toSparkline(n: number): result<string, distributionError> {
return Constructors_toSparkline({ env: this.env }, this.t, n);
}
algebraicAdd(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_algebraicAdd({ env: this.env }, this.t, d2.t)
);
}
algebraicMultiply(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_algebraicMultiply({ env: this.env }, this.t, d2.t)
);
}
algebraicDivide(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_algebraicDivide({ env: this.env }, this.t, d2.t)
);
}
algebraicSubtract(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_algebraicSubtract({ env: this.env }, this.t, d2.t)
);
}
algebraicLogarithm(
d2: Distribution
): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_algebraicLogarithm({ env: this.env }, this.t, d2.t)
);
}
algebraicPower(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_algebraicPower({ env: this.env }, this.t, d2.t)
);
}
pointwiseAdd(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_pointwiseAdd({ env: this.env }, this.t, d2.t)
);
}
pointwiseMultiply(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_pointwiseMultiply({ env: this.env }, this.t, d2.t)
);
}
pointwiseDivide(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_pointwiseDivide({ env: this.env }, this.t, d2.t)
);
}
pointwiseSubtract(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_pointwiseSubtract({ env: this.env }, this.t, d2.t)
);
}
pointwiseLogarithm(
d2: Distribution
): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_pointwiseLogarithm({ env: this.env }, this.t, d2.t)
);
}
pointwisePower(d2: Distribution): result<Distribution, distributionError> {
return this.mapResultDist(
Constructors_pointwisePower({ env: this.env }, this.t, d2.t)
);
}
}

View File

@ -0,0 +1,155 @@
import * as _ from "lodash";
import {
mixedShape,
sampleSetDist,
genericDist,
samplingParams,
symbolicDist,
discreteShape,
continuousShape,
} from "../rescript/TypescriptInterface.gen";
import { Distribution } from "./distribution";
import { tagged, tag } from "./types";
// This file is here to compensate for genType not fully recursively converting types
// Raw rescript types.
export type rescriptExport =
| {
TAG: 0; // EvArray
_0: rescriptExport[];
}
| {
TAG: 1; // EvBool
_0: boolean;
}
| {
TAG: 2; // EvCall
_0: string;
}
| {
TAG: 3; // EvDistribution
_0: rescriptDist;
}
| {
TAG: 4; // EvNumber
_0: number;
}
| {
TAG: 5; // EvRecord
_0: { [key: string]: rescriptExport };
}
| {
TAG: 6; // EvString
_0: string;
}
| {
TAG: 7; // EvSymbol
_0: string;
};
type rescriptDist =
| { TAG: 0; _0: rescriptPointSetDist }
| { TAG: 1; _0: sampleSetDist }
| { TAG: 2; _0: symbolicDist };
type rescriptPointSetDist =
| {
TAG: 0; // Mixed
_0: mixedShape;
}
| {
TAG: 1; // Discrete
_0: discreteShape;
}
| {
TAG: 2; // ContinuousShape
_0: continuousShape;
};
export type squiggleExpression =
| tagged<"symbol", string>
| tagged<"string", string>
| tagged<"call", string>
| tagged<"array", squiggleExpression[]>
| tagged<"boolean", boolean>
| tagged<"distribution", Distribution>
| tagged<"number", number>
| tagged<"record", { [key: string]: squiggleExpression }>;
export function convertRawToTypescript(
result: rescriptExport,
sampEnv: samplingParams
): squiggleExpression {
switch (result.TAG) {
case 0: // EvArray
return tag(
"array",
result._0.map((x) => convertRawToTypescript(x, sampEnv))
);
case 1: // EvBool
return tag("boolean", result._0);
case 2: // EvCall
return tag("call", result._0);
case 3: // EvDistribution
return tag(
"distribution",
new Distribution(
convertRawDistributionToGenericDist(result._0),
sampEnv
)
);
case 4: // EvNumber
return tag("number", result._0);
case 5: // EvRecord
return tag(
"record",
_.mapValues(result._0, (x) => convertRawToTypescript(x, sampEnv))
);
case 6: // EvString
return tag("string", result._0);
case 7: // EvSymbol
return tag("symbol", result._0);
}
}
function convertRawDistributionToGenericDist(
result: rescriptDist
): genericDist {
switch (result.TAG) {
case 0: // Point Set Dist
switch (result._0.TAG) {
case 0: // Mixed
return tag("PointSet", tag("Mixed", result._0._0));
case 1: // Discrete
return tag("PointSet", tag("Discrete", result._0._0));
case 2: // Continuous
return tag("PointSet", tag("Continuous", result._0._0));
}
case 1: // Sample Set Dist
return tag("SampleSet", result._0);
case 2: // Symbolic Dist
return tag("Symbolic", result._0);
}
}
export type jsValue =
| string
| number
| jsValue[]
| { [key: string]: jsValue }
| boolean;
export function jsValueToBinding(value: jsValue): rescriptExport {
if (typeof value === "boolean") {
return { TAG: 1, _0: value as boolean };
} else if (typeof value === "string") {
return { TAG: 6, _0: value as string };
} else if (typeof value === "number") {
return { TAG: 4, _0: value as number };
} else if (Array.isArray(value)) {
return { TAG: 0, _0: value.map(jsValueToBinding) };
} else {
// Record
return { TAG: 5, _0: _.mapValues(value, jsValueToBinding) };
}
}

View File

@ -0,0 +1,30 @@
export type result<a, b> =
| {
tag: "Ok";
value: a;
}
| {
tag: "Error";
value: b;
};
export function resultMap<a, b, c>(
r: result<a, c>,
mapFn: (x: a) => b
): result<b, c> {
if (r.tag === "Ok") {
return { tag: "Ok", value: mapFn(r.value) };
} else {
return r;
}
}
export function Ok<a, b>(x: a): result<a, b> {
return { tag: "Ok", value: x };
}
export type tagged<a, b> = { tag: a; value: b };
export function tag<a, b>(x: a, y: b): tagged<a, b> {
return { tag: x, value: y };
}

View File

@ -158,6 +158,16 @@ let rec run = (~env, functionCallInfo: functionCallInfo): outputType => {
->GenericDist.toPointSet(~xyPointLength, ~sampleCount, ())
->E.R2.fmap(r => Dist(PointSet(r)))
->OutputLocal.fromResult
| ToDist(Scale(#Logarithm, f)) =>
dist
->GenericDist.pointwiseCombinationFloat(~toPointSetFn, ~algebraicCombination=#Logarithm, ~f)
->E.R2.fmap(r => Dist(r))
->OutputLocal.fromResult
| ToDist(Scale(#Power, f)) =>
dist
->GenericDist.pointwiseCombinationFloat(~toPointSetFn, ~algebraicCombination=#Power, ~f)
->E.R2.fmap(r => Dist(r))
->OutputLocal.fromResult
| ToDistCombination(Algebraic(_), _, #Float(_)) => GenDistError(NotYetImplemented)
| ToDistCombination(Algebraic(strategy), arithmeticOperation, #Dist(t2)) =>
dist
@ -193,6 +203,12 @@ let rec run = (~env, functionCallInfo: functionCallInfo): outputType => {
->GenericDist.mixture(~scaleMultiplyFn=scaleMultiply, ~pointwiseAddFn=pointwiseAdd)
->E.R2.fmap(r => Dist(r))
->OutputLocal.fromResult
| FromSamples(xs) =>
xs
->SampleSetDist.make
->E.R2.errMap(x => DistributionTypes.SampleSetError(x))
->E.R2.fmap(x => x->DistributionTypes.SampleSet->Dist)
->OutputLocal.fromResult
}
}
@ -234,6 +250,7 @@ module Constructors = {
let logScore = (~env, dist1, dist2) => C.logScore(dist1, dist2)->run(~env)->toFloatR
let toPointSet = (~env, dist) => C.toPointSet(dist)->run(~env)->toDistR
let toSampleSet = (~env, dist, n) => C.toSampleSet(dist, n)->run(~env)->toDistR
let fromSamples = (~env, xs) => C.fromSamples(xs)->run(~env)->toDistR
let truncate = (~env, dist, leftCutoff, rightCutoff) =>
C.truncate(dist, leftCutoff, rightCutoff)->run(~env)->toDistR
let inspect = (~env, dist) => C.inspect(dist)->run(~env)->toDistR

View File

@ -63,6 +63,8 @@ module Constructors: {
@genType
let toSampleSet: (~env: env, genericDist, int) => result<genericDist, error>
@genType
let fromSamples: (~env: env, SampleSetDist.t) => result<genericDist, error>
@genType
let truncate: (~env: env, genericDist, option<float>, option<float>) => result<genericDist, error>
@genType
let inspect: (~env: env, genericDist) => result<genericDist, error>

View File

@ -11,7 +11,7 @@ type error =
| NotYetImplemented
| Unreachable
| DistributionVerticalShiftIsInvalid
| TooFewSamples
| SampleSetError(SampleSetDist.sampleSetError)
| ArgumentError(string)
| OperationError(Operation.Error.t)
| PointSetConversionError(SampleSetDist.pointsetConversionError)
@ -35,7 +35,8 @@ module Error = {
| DistributionVerticalShiftIsInvalid => "Distribution Vertical Shift is Invalid"
| ArgumentError(s) => `Argument Error ${s}`
| LogarithmOfDistributionError(s) => `Logarithm of input error: ${s}`
| TooFewSamples => "Too Few Samples"
| SampleSetError(TooFewSamples) => "Too Few Samples"
| SampleSetError(NonNumericInput(err)) => `Found a non-number in input: ${err}`
| OperationError(err) => Operation.Error.toString(err)
| PointSetConversionError(err) => SampleSetDist.pointsetConversionErrorToString(err)
| SparklineError(err) => PointSetTypes.sparklineErrorToString(err)
@ -47,10 +48,7 @@ module Error = {
let resultStringToResultError: result<'a, string> => result<'a, error> = n =>
n->E.R2.errMap(r => r->fromString)
let sampleErrorToDistErr = (err: SampleSetDist.sampleSetError): error =>
switch err {
| TooFewSamples => TooFewSamples
}
let sampleErrorToDistErr = (err: SampleSetDist.sampleSetError): error => SampleSetError(err)
}
@genType
@ -68,12 +66,19 @@ module DistributionOperation = {
| #Pdf(float)
| #Mean
| #Sample
| #IntegralSum
]
type toScaleFn = [
| #Power
| #Logarithm
]
type toDist =
| Normalize
| ToPointSet
| ToSampleSet(int)
| Scale(toScaleFn, float)
| Truncate(option<float>, option<float>)
| Inspect
@ -102,6 +107,7 @@ module DistributionOperation = {
type genericFunctionCallInfo =
| FromDist(fromDist, genericDist)
| FromFloat(fromDist, float)
| FromSamples(array<float>)
| Mixture(array<(genericDist, float)>)
let distCallToString = (distFunction: fromDist): string =>
@ -117,6 +123,8 @@ module DistributionOperation = {
| ToDist(ToSampleSet(r)) => `toSampleSet(${E.I.toString(r)})`
| ToDist(Truncate(_, _)) => `truncate`
| ToDist(Inspect) => `inspect`
| ToDist(Scale(#Power, r)) => `scalePower(${E.Float.toFixed(r)})`
| ToDist(Scale(#Logarithm, r)) => `scaleLog(${E.Float.toFixed(r)})`
| ToString(ToString) => `toString`
| ToString(ToSparkline(n)) => `toSparkline(${E.I.toString(n)})`
| ToBool(IsNormalized) => `isNormalized`
@ -128,6 +136,7 @@ module DistributionOperation = {
switch d {
| FromDist(f, _) | FromFloat(f, _) => distCallToString(f)
| Mixture(_) => `mixture`
| FromSamples(_) => `fromSamples`
}
}
module Constructors = {
@ -144,9 +153,12 @@ module Constructors = {
let isNormalized = (dist): t => FromDist(ToBool(IsNormalized), dist)
let toPointSet = (dist): t => FromDist(ToDist(ToPointSet), dist)
let toSampleSet = (dist, r): t => FromDist(ToDist(ToSampleSet(r)), dist)
let fromSamples = (xs): t => FromSamples(xs)
let truncate = (dist, left, right): t => FromDist(ToDist(Truncate(left, right)), dist)
let inspect = (dist): t => FromDist(ToDist(Inspect), dist)
let logScore = (dist1, dist2): t => FromDist(ToScore(LogScore(dist2)), dist1)
let scalePower = (dist, n): t => FromDist(ToDist(Scale(#Power, n)), dist)
let scaleLogarithm = (dist, n): t => FromDist(ToDist(Scale(#Logarithm, n)), dist)
let toString = (dist): t => FromDist(ToString(ToString), dist)
let toSparkline = (dist, n): t => FromDist(ToString(ToSparkline(n)), dist)
let algebraicAdd = (dist1, dist2: genericDist): t => FromDist(

View File

@ -69,10 +69,13 @@ let logScore = (t1, t2, ~toPointSetFn: toPointSetFn): result<float, error> => {
let toFloatOperation = (
t,
~toPointSetFn: toPointSetFn,
~distToFloatOperation: Operation.distToFloatOperation,
~distToFloatOperation: DistributionTypes.DistributionOperation.toFloat,
) => {
switch distToFloatOperation {
| #IntegralSum => Ok(integralEndY(t))
| (#Pdf(_) | #Cdf(_) | #Inv(_) | #Mean | #Sample) as op => {
let trySymbolicSolution = switch (t: t) {
| Symbolic(r) => SymbolicDist.T.operate(distToFloatOperation, r)->E.R.toOption
| Symbolic(r) => SymbolicDist.T.operate(op, r)->E.R.toOption
| _ => None
}
@ -88,7 +91,9 @@ let toFloatOperation = (
| None =>
switch trySampleSetSolution {
| Some(r) => Ok(r)
| None => toPointSetFn(t)->E.R2.fmap(PointSetDist.operate(distToFloatOperation))
| None => toPointSetFn(t)->E.R2.fmap(PointSetDist.operate(op))
}
}
}
}
}

View File

@ -20,7 +20,7 @@ let isNormalized: t => bool
let toFloatOperation: (
t,
~toPointSetFn: toPointSetFn,
~distToFloatOperation: Operation.distToFloatOperation,
~distToFloatOperation: DistributionTypes.DistributionOperation.toFloat,
) => result<float, error>
let logScore: (t, t, ~toPointSetFn: toPointSetFn) => result<float, error>

View File

@ -156,8 +156,10 @@ let reduce = (
~integralSumCachesFn: (float, float) => option<float>=(_, _) => None,
fn: (float, float) => result<float, 'e>,
continuousShapes,
): result<t, 'e> =>
continuousShapes |> E.A.R.foldM(combinePointwise(~integralSumCachesFn, fn), empty)
): result<t, 'e> => {
let merge = combinePointwise(~integralSumCachesFn, fn)
continuousShapes |> E.A.R.foldM(merge, empty)
}
let mapYResult = (
~integralSumCacheFn=_ => None,

View File

@ -37,7 +37,7 @@ let combinePointwise = (
~fn=(a, b) => Ok(a +. b),
t1: PointSetTypes.discreteShape,
t2: PointSetTypes.discreteShape,
): PointSetTypes.discreteShape => {
): result<PointSetTypes.discreteShape, 'e> => {
let combinedIntegralSum = Common.combineIntegralSums(
integralSumCachesFn,
t1.integralSumCache,
@ -54,11 +54,17 @@ let combinePointwise = (
t1.xyShape,
t2.xyShape,
)->E.R.toExn("Addition operation should never fail", _),
)
)->Ok
}
let reduce = (~integralSumCachesFn=(_, _) => None, discreteShapes): PointSetTypes.discreteShape =>
discreteShapes |> E.A.fold_left(combinePointwise(~integralSumCachesFn), empty)
let reduce = (
~integralSumCachesFn=(_, _) => None,
fn: (float, float) => result<float, 'e>,
discreteShapes: array<PointSetTypes.discreteShape>,
): result<t, 'e> => {
let merge = combinePointwise(~integralSumCachesFn, fn)
discreteShapes |> E.A.R.foldM(merge, empty)
}
let updateIntegralSumCache = (integralSumCache, t: t): t => {
...t,

View File

@ -350,3 +350,43 @@ let combineAlgebraically = (op: Operation.convolutionOperation, t1: t, t2: t): t
integralCache: None,
}
}
let combinePointwise = (
~integralSumCachesFn=(_, _) => None,
~integralCachesFn=(_, _) => None,
fn: (float, float) => result<float, 'e>,
t1: t,
t2: t,
): result<t, 'e> => {
let reducedDiscrete =
[t1, t2]
|> E.A.fmap(toDiscrete)
|> E.A.O.concatSomes
|> Discrete.reduce(~integralSumCachesFn, fn)
let reducedContinuous =
[t1, t2]
|> E.A.fmap(toContinuous)
|> E.A.O.concatSomes
|> Continuous.reduce(~integralSumCachesFn, fn)
let combinedIntegralSum = Common.combineIntegralSums(
integralSumCachesFn,
t1.integralSumCache,
t2.integralSumCache,
)
let combinedIntegral = Common.combineIntegrals(
integralCachesFn,
t1.integralCache,
t2.integralCache,
)
E.R.merge(reducedContinuous, reducedDiscrete)->E.R2.fmap(((continuous, discrete)) =>
make(
~integralSumCache=combinedIntegralSum,
~integralCache=combinedIntegral,
~discrete,
~continuous,
)
)
}

View File

@ -84,7 +84,12 @@ let combinePointwise = (
m2,
)->E.R2.fmap(x => PointSetTypes.Continuous(x))
| (Discrete(m1), Discrete(m2)) =>
Ok(PointSetTypes.Discrete(Discrete.combinePointwise(~integralSumCachesFn, m1, m2)))
Discrete.combinePointwise(
~integralSumCachesFn,
fn,
m1,
m2,
)->E.R2.fmap(x => PointSetTypes.Discrete(x))
| (m1, m2) =>
Mixed.combinePointwise(
~integralSumCachesFn,

View File

@ -1,11 +1,12 @@
@genType
module Error = {
@genType
type sampleSetError = TooFewSamples
type sampleSetError = TooFewSamples | NonNumericInput(string)
let sampleSetErrorToString = (err: sampleSetError): string =>
switch err {
| TooFewSamples => "Too few samples when constructing sample set"
| NonNumericInput(err) => `Found a non-number in input: ${err}`
}
@genType

View File

@ -1,27 +1,30 @@
//The math here was taken from https://github.com/jasondavies/science.js/blob/master/src/stats/SampleSetDist_Bandwidth.js
let {iqr_percentile, nrd0_lo_denominator, one, nrd0_coef, nrd_coef, nrd_fractionalPower} = module(
MagicNumbers.SampleSetBandwidth
)
let len = x => E.A.length(x) |> float_of_int
let iqr = x => Jstat.percentile(x, 0.75, true) -. Jstat.percentile(x, 0.25, true)
let iqr = x =>
Jstat.percentile(x, iqr_percentile, true) -. Jstat.percentile(x, 1.0 -. iqr_percentile, true)
// Silverman, B. W. (1986) Density Estimation. London: Chapman and Hall.
let nrd0 = x => {
let hi = Js_math.sqrt(Jstat.variance(x))
let lo = Js_math.minMany_float([hi, iqr(x) /. 1.34])
let lo = Js_math.minMany_float([hi, iqr(x) /. nrd0_lo_denominator])
let e = Js_math.abs_float(x[1])
let lo' = switch (lo, hi, e) {
| (lo, _, _) if !Js.Float.isNaN(lo) => lo
| (_, hi, _) if !Js.Float.isNaN(hi) => hi
| (_, _, e) if !Js.Float.isNaN(e) => e
| _ => 1.0
| _ => one
}
0.9 *. lo' *. Js.Math.pow_float(~base=len(x), ~exp=-0.2)
nrd0_coef *. lo' *. Js.Math.pow_float(~base=len(x), ~exp=nrd_fractionalPower)
}
// Scott, D. W. (1992) Multivariate Density Estimation: Theory, Practice, and Visualization. Wiley.
let nrd = x => {
let h = iqr(x) /. 1.34
1.06 *.
let h = iqr(x) /. nrd0_lo_denominator
nrd_coef *.
Js.Math.min_float(Js.Math.sqrt(Jstat.variance(x)), h) *.
Js.Math.pow_float(~base=len(x), ~exp=-1.0 /. 5.0)
Js.Math.pow_float(~base=len(x), ~exp=nrd_fractionalPower)
}

View File

@ -35,3 +35,16 @@ module ToPointSet = {
*/
let minDiscreteToKeep = samples => max(20, E.A.length(samples) / 50)
}
module SampleSetBandwidth = {
// Silverman, B. W. (1986) Density Estimation. London: Chapman and Hall.
// Scott, D. W. (1992) Multivariate Density Estimation: Theory, Practice, and Visualization. Wiley.
let iqr_percentile = 0.75
let iqr_percentile_complement = 1.0 -. iqr_percentile
let nrd0_lo_denominator = 1.34
let one = 1.0
let nrd0_coef = 0.9
let nrd_coef = 1.06
let nrd_fractionalPower = -0.2
}

View File

@ -132,6 +132,7 @@ module Helpers = {
| Error(err) => GenDistError(ArgumentError(err))
}
}
| Some(EvNumber(_))
| Some(EvDistribution(_)) =>
switch parseDistributionArray(args) {
| Ok(distributions) => mixtureWithDefaultWeights(distributions)
@ -197,6 +198,7 @@ let dispatchToGenericOutput = (call: ExpressionValue.functionCall): option<
->SymbolicConstructors.symbolicResultToOutput
| ("sample", [EvDistribution(dist)]) => Helpers.toFloatFn(#Sample, dist)
| ("mean", [EvDistribution(dist)]) => Helpers.toFloatFn(#Mean, dist)
| ("integralSum", [EvDistribution(dist)]) => Helpers.toFloatFn(#IntegralSum, dist)
| ("toString", [EvDistribution(dist)]) => Helpers.toStringFn(ToString, dist)
| ("toSparkline", [EvDistribution(dist)]) => Helpers.toStringFn(ToSparkline(20), dist)
| ("toSparkline", [EvDistribution(dist), EvNumber(n)]) =>
@ -215,6 +217,15 @@ let dispatchToGenericOutput = (call: ExpressionValue.functionCall): option<
)
| ("isNormalized", [EvDistribution(dist)]) => Helpers.toBoolFn(IsNormalized, dist)
| ("toPointSet", [EvDistribution(dist)]) => Helpers.toDistFn(ToPointSet, dist)
| ("scaleLog", [EvDistribution(dist)]) =>
Helpers.toDistFn(Scale(#Logarithm, MagicNumbers.Math.e), dist)
| ("scaleLog10", [EvDistribution(dist)]) => Helpers.toDistFn(Scale(#Logarithm, 10.0), dist)
| ("scaleLog", [EvDistribution(dist), EvNumber(float)]) =>
Helpers.toDistFn(Scale(#Logarithm, float), dist)
| ("scalePow", [EvDistribution(dist), EvNumber(float)]) =>
Helpers.toDistFn(Scale(#Power, float), dist)
| ("scaleExp", [EvDistribution(dist)]) =>
Helpers.toDistFn(Scale(#Power, MagicNumbers.Math.e), dist)
| ("cdf", [EvDistribution(dist), EvNumber(float)]) => Helpers.toFloatFn(#Cdf(float), dist)
| ("pdf", [EvDistribution(dist), EvNumber(float)]) => Helpers.toFloatFn(#Pdf(float), dist)
| ("inv", [EvDistribution(dist), EvNumber(float)]) => Helpers.toFloatFn(#Inv(float), dist)
@ -222,6 +233,14 @@ let dispatchToGenericOutput = (call: ExpressionValue.functionCall): option<
Helpers.toDistFn(ToSampleSet(Belt.Int.fromFloat(float)), dist)
| ("toSampleSet", [EvDistribution(dist)]) =>
Helpers.toDistFn(ToSampleSet(MagicNumbers.Environment.defaultSampleCount), dist)
| ("fromSamples", [EvArray(inputArray)]) => {
let _wrapInputErrors = x => SampleSetDist.NonNumericInput(x)
let parsedArray = Helpers.parseNumberArray(inputArray)->E.R2.errMap(_wrapInputErrors)
switch parsedArray {
| Ok(array) => runGenericOperation(FromSamples(array))
| Error(e) => GenDistError(SampleSetError(e))
}->Some
}
| ("inspect", [EvDistribution(dist)]) => Helpers.toDistFn(Inspect, dist)
| ("truncateLeft", [EvDistribution(dist), EvNumber(float)]) =>
Helpers.toDistFn(Truncate(Some(float), None), dist)

View File

@ -13,6 +13,12 @@ type samplingParams = DistributionOperation.env
@genType
type genericDist = DistributionTypes.genericDist
@genType
type sampleSetDist = SampleSetDist.t
@genType
type symbolicDist = SymbolicDistTypes.symbolicDist
@genType
type distributionError = DistributionTypes.error
@ -34,6 +40,12 @@ let evaluate = Reducer.evaluate
@genType
let evaluateUsingExternalBindings = Reducer.evaluateUsingExternalBindings
@genType
let evaluatePartialUsingExternalBindings = Reducer.evaluatePartialUsingExternalBindings
@genType
type externalBindings = Reducer.externalBindings
@genType
type expressionValue = ReducerInterface_ExpressionValue.expressionValue

View File

@ -289,6 +289,13 @@ module R = {
| Ok(r) => r->Ok
| Error(x) => x->f->Error
}
//I'm not sure what to call this.
let unify = (a: result<'a, 'b>, c: 'b => 'a): 'a =>
switch a {
| Ok(x) => x
| Error(x) => c(x)
}
}
module R2 = {
@ -307,6 +314,8 @@ module R2 = {
| Ok(x) => x->Ok
| Error(x) => x->f->Error
}
let toExn = (a, b) => R.toExn(b, a)
}
let safe_fn_of_string = (fn, s: string): option<'a> =>
@ -350,7 +359,7 @@ module JsDate = {
/* List */
module L = {
module Util = {
let eq = (a, b) => a == b
let eq = \"=="
}
let fmap = List.map
let get = Belt.List.get

View File

@ -98,6 +98,19 @@ bound `a`, mode `b` and upper bound `c`.
Squiggle, when the context is right, automatically casts a float to a constant distribution.
## `fromSamples`
The last distribution constructor takes an array of samples and constructs a sample set distribution.
<SquiggleEditor initialSquiggleString="fromSamples([1,2,3,4,6,5,5,5])" />
#### Validity
For `fromSamples(xs)`,
- `xs.length > 5`
- Strictly every element of `xs` must be a number.
## Operating on distributions
Here are the ways we combine distributions.
@ -315,6 +328,16 @@ Or `PointSet` format
<SquiggleEditor initialSquiggleString="toPointSet(normal(5, 10))" />
### `toSampleSet` has two signatures
Above, we saw the unary `toSampleSet`, which uses an internal hardcoded number of samples. If you'd like to provide the number of samples, it has a binary signature as well (floored)
<SquiggleEditor initialSquiggleString="toSampleSet(0.1 to 1, 100.1)" />
#### Validity
- Second argument to `toSampleSet` must be a number.
## Normalization
Some distribution operations (like horizontal shift) return an unnormalized distriibution.
@ -333,18 +356,6 @@ We provide a predicate `isNormalized`, for when we have simple control flow
- Input to `isNormalized` must be a dist
## Convert any distribution to a sample set distribution
`toSampleSet` has two signatures
It is unary when you use an internal hardcoded number of samples
<SquiggleEditor initialSquiggleString="toSampleSet(0.1 to 1)" />
And binary when you provide a number of samples (floored)
<SquiggleEditor initialSquiggleString="toSampleSet(0.1 to 1, 100)" />
## `inspect`
You may like to debug by right clicking your browser and using the _inspect_ functionality on the webpage, and viewing the _console_ tab. Then, wrap your squiggle output with `inspect` to log an internal representation.

975
yarn.lock

File diff suppressed because it is too large Load Diff