All pages
Docs · IndexMarkdown
SexpGPU usage
SexpGPU is a Common Lisp for stating machine learning experiments, a compiler that turns one experiment file into a checked IR, and a Rust runtime that trains it on GPUs. Each page answers one question and links the rest.
Every page is also served as markdown at /docs/<page>.md, and /llms.txt lists every page.
Start here
Why SexpGPU
what goes wrong in ML research code and what a one-file experiment fixes
Install
getting the binary, what a machine needs to run it
Quickstart
check, explain and run a first experiment in five commands
How it works
the file, the compile-time Lisp, the IR and the fixed training loop
Working as an agent
the loop, the machine-readable outputs, and the rules that keep a run honest
The run file
The run file
the contract names, their order, and the smallest file that trains
Modules and require
the standard modules, relative libraries, scopes and imports
Knobs, variants and sweeps
everything a run may vary, declared in the file, selected before it is read
Loaders and prepare
sources, fields, stages, batches,
prepare and countersObjective and evaluate
the loss that is optimized, the metrics that are reported
Evaluation passes
defeval: named loaders with a cadenceOptimizer groups
configuring an optimizer, selectors, per-group hyperparameters and optimizers
Curriculum and ctx
moving between loader stages, and every
ctx keydefrun
steps, microbatches, precision, seed, checkpoints, diagnostics sampling
The language
It is Common Lisp
the implemented subset, the reader, the deliberate differences, evaluation limits
Special forms
every form the evaluator treats specially, with its shape
Functions and bindings
let, setq, lambda lists, apply, flet, labelsTruth, iteration and values
what is true,
dotimes, dolist, multiple values, stagingMacros and symbols
capture, gensyms,
once-only, macroexpand, generated declarationsBuiltins
every compile-time builtin on ordinary values
The prelude
loaded before every file: every macro and function in it
Tensor operations
shapes, dtypes, broadcasting and every tensor builtin
Models and optimizers
Models and parameters
defmodel, defparam, parameter paths, weight tyingPrecision and numerics
:bf16, with-numerics, f32 islands and master weightsWriting optimizers
defoptimizer, defstate, optimizer-update, schedulessexpgpu/nn
the standard neural network module, every definition
sexpgpu/optim
the standard optimizer module, every definition
The command line
The command line
every verb, the selection flags, standard streams, exit codes
check and watch
diagnostics in under a second, again on every save
explain, diff, ir and eval
what the compiler made of a file, the CUDA lowering report, and what changed between two
new, fmt and variants
starting a run from another, the one layout, the declared selection
run
training: flags, output, the slug, stopping
sweep
every point of a declared sweep, in order
bundle and remote runs
one directory that runs anywhere, preemptible jobs, and reading a preempted run
doctor
whether a run can start on this machine
Devices
the CPU interpreter, the CUDA device, its memory plan, several GPUs and nodes
Environment
every variable, S3 credentials, and what wins over what
Observing a run
Metrics
targets,
metric, runtime metadata, the series every run reportsDiagnostics
numbers that cost nothing until selected; selection, sampling, the built-in set
Writing diagnostics
diagnostic, reducers, optimizer and gradient diagnosticsJSONL events
the event kinds, annotations and the
open metadataThe status file
the one file to poll, and the final summary
Checkpoints and resume
what a checkpoint holds and
sexpgpu checkpoint, --resume latest, signals, the finite guard