# SexpGPU

> S-exp GPU: ML research at the speed of thought. The world's first ML framework designed for agents: a Common Lisp for stating machine learning experiments, a compiler that turns one experiment file into a checked IR, and a Rust runtime that trains it on GPUs at torch.compile's step time or faster on nine of the eleven measured models.

Every link below is a markdown page. Any page on this site also answers with markdown when asked for `Accept: text/markdown` or fetched with curl. https://sexpgpu.041.io/llms-full.txt is every page in one file. Start with "Working as an agent".

## Start here

- [Why SexpGPU](https://sexpgpu.041.io/docs/why.md): what goes wrong in ML research code and what a one-file experiment fixes
- [Install](https://sexpgpu.041.io/docs/install.md): getting the binary, what a machine needs to run it
- [Quickstart](https://sexpgpu.041.io/docs/quickstart.md): check, explain and run a first experiment in five commands
- [How it works](https://sexpgpu.041.io/docs/how-it-works.md): the file, the compile-time Lisp, the IR and the fixed training loop
- [Working as an agent](https://sexpgpu.041.io/docs/agents.md): the loop, the machine-readable outputs, and the rules that keep a run honest

## The run file

- [The run file](https://sexpgpu.041.io/docs/run-file.md): the contract names, their order, and the smallest file that trains
- [Modules and require](https://sexpgpu.041.io/docs/require.md): the standard modules, relative libraries, scopes and imports
- [Knobs, variants and sweeps](https://sexpgpu.041.io/docs/knobs.md): everything a run may vary, declared in the file, selected before it is read
- [Loaders and prepare](https://sexpgpu.041.io/docs/data.md): sources, fields, stages, batches, `prepare` and counters
- [Objective and evaluate](https://sexpgpu.041.io/docs/objective.md): the loss that is optimized, the metrics that are reported
- [Evaluation passes](https://sexpgpu.041.io/docs/evaluation.md): `defeval`: named loaders with a cadence
- [Optimizer groups](https://sexpgpu.041.io/docs/optimizer-groups.md): configuring an optimizer, selectors, per-group hyperparameters and optimizers
- [Curriculum and ctx](https://sexpgpu.041.io/docs/curriculum.md): moving between loader stages, and every `ctx` key
- [defrun](https://sexpgpu.041.io/docs/defrun.md): steps, microbatches, precision, seed, checkpoints, diagnostics sampling

## The language

- [It is Common Lisp](https://sexpgpu.041.io/docs/lisp.md): the implemented subset, the reader, the deliberate differences, evaluation limits
- [Special forms](https://sexpgpu.041.io/docs/special-forms.md): every form the evaluator treats specially, with its shape
- [Functions and bindings](https://sexpgpu.041.io/docs/functions.md): `let`, `setq`, lambda lists, `apply`, `flet`, `labels`
- [Truth, iteration and values](https://sexpgpu.041.io/docs/control.md): what is true, `dotimes`, `dolist`, multiple values, staging
- [Macros and symbols](https://sexpgpu.041.io/docs/macros.md): capture, gensyms, `once-only`, `macroexpand`, generated declarations
- [Builtins](https://sexpgpu.041.io/docs/builtins.md): every compile-time builtin on ordinary values
- [The prelude](https://sexpgpu.041.io/docs/prelude.md): loaded before every file: every macro and function in it
- [Tensor operations](https://sexpgpu.041.io/docs/tensors.md): shapes, dtypes, broadcasting and every tensor builtin

## Models and optimizers

- [Models and parameters](https://sexpgpu.041.io/docs/models.md): `defmodel`, `defparam`, parameter paths, weight tying
- [Precision and numerics](https://sexpgpu.041.io/docs/numerics.md): `:bf16`, `with-numerics`, `f32` islands and master weights
- [Writing optimizers](https://sexpgpu.041.io/docs/optimizers.md): `defoptimizer`, `defstate`, `optimizer-update`, schedules
- [sexpgpu/nn](https://sexpgpu.041.io/docs/nn.md): the standard neural network module, every definition
- [sexpgpu/optim](https://sexpgpu.041.io/docs/optim.md): the standard optimizer module, every definition

## The command line

- [The command line](https://sexpgpu.041.io/docs/cli.md): every verb, the selection flags, standard streams, exit codes
- [check and watch](https://sexpgpu.041.io/docs/check.md): diagnostics in under a second, again on every save
- [explain, diff, ir and eval](https://sexpgpu.041.io/docs/explain.md): what the compiler made of a file, the CUDA lowering report, and what changed between two
- [new, fmt and variants](https://sexpgpu.041.io/docs/new.md): starting a run from another, the one layout, the declared selection
- [run](https://sexpgpu.041.io/docs/run.md): training: flags, output, the slug, stopping
- [sweep](https://sexpgpu.041.io/docs/sweep.md): every point of a declared sweep, in order
- [bundle and remote runs](https://sexpgpu.041.io/docs/bundle.md): one directory that runs anywhere, preemptible jobs, and reading a preempted run
- [doctor](https://sexpgpu.041.io/docs/doctor.md): whether a run can start on this machine
- [Devices](https://sexpgpu.041.io/docs/devices.md): the CPU interpreter, the CUDA device, its memory plan, several GPUs and nodes
- [Environment](https://sexpgpu.041.io/docs/environment.md): every variable, S3 credentials, and what wins over what

## Observing a run

- [Metrics](https://sexpgpu.041.io/docs/metrics.md): targets, `metric`, runtime metadata, the series every run reports
- [Diagnostics](https://sexpgpu.041.io/docs/diagnostics.md): numbers that cost nothing until selected; selection, sampling, the built-in set
- [Writing diagnostics](https://sexpgpu.041.io/docs/writing-diagnostics.md): `diagnostic`, reducers, optimizer and gradient diagnostics
- [JSONL events](https://sexpgpu.041.io/docs/events.md): the event kinds, annotations and the `open` metadata
- [The status file](https://sexpgpu.041.io/docs/status.md): the one file to poll, and the final summary
- [Checkpoints and resume](https://sexpgpu.041.io/docs/checkpoints.md): what a checkpoint holds and `sexpgpu checkpoint`, `--resume latest`, signals, the finite guard

## Reference

- [Diagnostic codes](https://sexpgpu.041.io/docs/errors.md): every `E-*` and `W-*` code, what it means and the usual fix

## Optional

- [Landing page](https://sexpgpu.041.io/index.md): performance against PyTorch, the command line, code examples