All pages
The command line
One binary, sexpgpu, does four things: author a run (new, fmt,
variants, watch), check it (check, explain, diff, ir, eval),
run it (run, sweep) and pack it for another machine (bundle).
doctor says whether a run can start here, and checkpoint what a
checkpoint holds. There is no run management: a
run is one process, and whatever starts it (a terminal, a job runner, a
scheduler) owns the rest.
Verbs
| verb | arguments | what it does | page |
|---|---|---|---|
check | <file.sx> [--json] | compile and report diagnostics | check |
watch | <file.sx> | check again on every change, until Ctrl-C | check |
explain | <file.sx> [--device cuda] [--json] | what the compiler made of the file, and with a device what the executor makes of it | explain |
diff | <a.sx> <b.sx> [--json] | what changed between two runs, or two selections of one | explain |
ir | <file.sx> | the experiment IR as JSON | explain |
eval | <file.sx> | the value of each top-level form | explain |
fmt | <file.sx>... [--check] [--stdout] | rewrite files in the one fixed layout | new |
variants | <file.sx> | the knobs, variants and sweeps the file declares | new |
new | <target.sx> --from <file> [--knob k=v]... | start a run from a template or another run | new |
run | <file.sx> | run the experiment | run |
sweep | <file.sx> <sweep> [--dry-run] [--only N] | run every point of a declared sweep | sweep |
bundle | <file.sx> -o <dir> [--binary <sexpgpu>] | one directory that runs the experiment anywhere | bundle |
checkpoint | <dir>|s3://bucket/prefix/step-<n> | what a checkpoint holds: its experiment, step and tensors | checkpoints |
doctor | — | can a run start here, and with what | doctor |
The selection
check, explain, diff, watch, ir, run and bundle take:
| flag | meaning |
|---|---|
--variant <name> | select a declared variant |
--set <knob>=<value> | set a knob, repeatable |
--diagnostics <glob,glob> | replaces defrun :diagnostics; '' selects none. See diagnostics |
For diff the selection applies to b only. sweep takes all three:
--set pins a knob across every point (pinning an axis of the sweep exits
2), --variant applies when the sweep names no variant, --diagnostics
applies to every point. See knobs.
Standard streams
Standard output carries a verb's answer: JSON, IR, formatted source, a
table. Standard error carries what a run says while it works: diagnostics,
progress, the timing block, the done: block. run keeps standard output
free unless asked for data: --json puts the summary there, and
--metrics stdout makes it the metrics stream. With both, the summary is
the stream's last line.
Exit codes
The same for every verb:
| code | when |
|---|---|
0 | it worked |
1 | the thing failed: diagnostics (an unknown --variant or --set knob among them), a run that could not start or died, a fmt --check that would rewrite, a doctor that says a run cannot start, a bundle whose data is not on this machine or whose run would not read its bundled data |
2 | the command line was wrong: an unknown verb, a missing argument, a --set without =, a --steps or --eval-every that is not a whole number, a sweep the file does not declare or an --only past its end, a pinned sweep axis, a --knob the file does not declare, a new or bundle target that exists |
128 + n | run stopped by signal n: 129 SIGHUP, 130 SIGINT, 143 SIGTERM. Any other verb whose standard output closes early, sexpgpu ir x.sx | head, ends by SIGPIPE, 141, as a Unix tool does; run and sweep fail on the write instead, with the status file saying so |
Paths
A path inside a .sx file never depends on the working directory: a
require resolves against the file that holds it, a loader path against
the experiment file. Only paths on the command line are relative to where
you are.
Related: environment, working as an agent.