new, fmt and variants

The authoring verbs: start a file from another, keep it in the one layout, and list what it lets a tool vary.

new

$ sexpgpu new my-run.sx --from base.sx --knob layers=8 --knob width=512
wrote my-run.sx from base.sx: 116 parameters, 300 steps

new <target.sx> --from <file> [--knob k=v]... copies a run or a template:

• rewrites the defknobs defaults named with --knob;
• notes the source and the date in a header;
• recomputes every relative require and loader path for the new directory;
• compiles the result before reporting it, so a successful new is a file that passes check.

--from is required, and a target that exists is refused (exit 2), as is a --knob the file does not declare. A copy that does not compile is written anyway, reported as a failure (exit 1) and left on disk to look at.

Copy only when the change is not a knob of an existing run; when it is, --set or a defvariant is the answer. See knobs.

fmt

$ sexpgpu fmt my-run.sx lib/mine.sx
$ sexpgpu fmt --check my-run.sx; echo $?
0

Rewrites each named file in the one layout there is; every non-flag argument is a file. --check prints each file that would change and exits 1 without touching anything. --stdout prints one file's formatted text instead of writing it.

The layout: two spaces, 80 columns. A form goes on one line when it fits with the brackets that close after it; otherwise its head decides. A body goes two spaces under its head. A call with keyword arguments takes one :key value pair per line; a call without them fills the line. A form that breaks puts one binding or one vector element per line. Comments stay where they were, and one blank line inside a body is kept. There is nothing to configure.

A rewrite is refused as E-FMT-001, and the file left as it was, unless formatting the result again leaves it alone and it reads as the same datums. That is a formatter bug, not a problem with your file.

variants

$ sexpgpu variants my-run.sx
knobs:
  lr      0.0015  peak learning rate of the block matrices, the default group
  layers  4       transformer blocks
  width   256     model width
  seq     1024    sequence length in tokens; a window holds seq + 1
  seed    1       the run seed; every random stream folds it in
variants:
  small  layers=2 width=128
  smoke  layers=2 width=128 seq=64
sweeps:
  seeds  3 points over seed

Everything the file lets a tool vary, on standard output: every knob with its default and doc, every variant with its overrides, every sweep with its size. See knobs, variants and sweeps.

Related: the command line, check.