S-exp GPU
All pages
Docs · Start hereMarkdown

Working as an agent

SexpGPU is built to be driven by an agent. Every verb answers in a form a program reads (JSON with --json where there is structure; a run's progress goes to standard error, its numbers to the metrics target), fails with a stable exit code, and never asks anything interactively.

The loop

sexpgpu new my-run.sx --from base.sx --knob width=512   # start from a run, never from nothing
sexpgpu check my-run.sx --json                          # diagnostics as JSON, exit 1 on any error
sexpgpu explain my-run.sx --json                        # compare against the plan
sexpgpu diff base.sx my-run.sx                          # the intended change and nothing else
sexpgpu run my-run.sx --variant smoke --steps 3 --device cpu   # it trains, locally
sexpgpu bundle my-run.sx -o out/my-run --binary ./sexpgpu-linux-cuda
  1. Plan in the file. Every quantity the experiment may vary is a knob; named configurations are variants; grids are sweeps. Change a knob with --set rather than editing the file. See knobs.
  2. Check until clean. check --json returns an array of diagnostics, each with code, message, resolved file:line:col, notes and call trace. Look the code up in diagnostic codes; most carry a fix. See check.
  3. Verify the science. explain states what will run: the selection with each knob's source, every parameter with its shape and update group, the optimizer's constants, records and counters per step, the evaluation cadence, precision and peak memory. Compare it with the plan before paying for a GPU. See explain.
  4. Verify the change. diff prints only what differs between two runs, or two selections of one, by section. A diff with more rows than the intended change is a mistake.
  5. Smoke locally. A smoke variant shrinks an LM-scale run so the interpreter can take real steps. See devices.
  6. Run remotely. bundle, copy, run.sh --resume latest with SEXPGPU_CHECKPOINT_DIR set. See bundle.

Reading a running or finished run

wantreadnever
where is it nowthe status file, SEXPGPU_STATUSgrep the log for a step number
every numberthe JSONL events, filter "kind":"metric"parse the progress lines
the final resultrun --json, one summary object on standard outputparse the done: block
did it failthe exit code, then state and error in the status filelook for "error" in text

The progress lines and the done: block are for humans and are not a contract.

Exit codes

0 worked, 1 the thing failed (diagnostics, a run that died, a doctor that says no), 2 the command line was wrong, 128 + n stopped by signal n. See the command line.

Rules that keep a run honest

  • One selection names one run: the slug is the file stem, the variant and every --set, so metrics never mix two experiments. See run.
  • --steps and --eval-every are recorded as overrides; a shortened run says so everywhere.
  • Diagnostics never change a training number. Turn them on freely; see diagnostics.
  • Format every file you write: sexpgpu fmt file.sx. See fmt.
  • Put --resume latest on a remote command line from the first launch, with a checkpoint location; latest needs one.

Reading these docs

Read only the pages you need: the index lists every page with one line each. On the website, fetch /docs/<page>.md for markdown, or send Accept: text/markdown to any page; /llms.txt is the index and /llms-full.txt is every page in one file.

Related: how it works, the command line.