# Evaluation passes

A pass is a named loader and a cadence. Every pass runs `eval-prepare` (or
`prepare`), the model and [`evaluate`](https://sexpgpu.041.io/docs/objective.md#evaluate) over its own
loader. A file declares as many passes as it needs.

```lisp
(defeval eval
  :loader (loader :sources [(files ["data/val.parquet"])]
                  :fields [(field :tokens :from "input_ids" :dtype :i32 :shape [1025])]
                  :batch-size 4)
  :every 25)

(defeval probe :loader val-loader :every 5 :batches 16)
```

| keyword | default | meaning |
|---|---|---|
| `:loader` | required | the records the pass reads; see [loaders](https://sexpgpu.041.io/docs/data.md) |
| `:every` | none | steps between runs; without it the pass runs only after the last step |
| `:batches` | the whole loader | read this many batches from the start; without it the loader must be finite |

## When a pass runs

- At step 0, every `:every` steps, and once more after the last step.
- Passes due at the same step run in the order the file declares them.
- A pass reopens its loader every time, so a limited pass reads the same
  records each time.
- `--eval-every N` replaces every pass's `:every` for one session; `0`
  leaves only the pass after the last step, so a curriculum never runs. See
  [run](https://sexpgpu.041.io/docs/run.md#flags).

## The pass named eval

The pass named `eval` is the one the summary, the status file, the progress
line and `sweep`'s table report; a file without one reports its first pass.
Every observation from a pass carries `pass` metadata, so `eval/loss` from
`eval` and from `probe` are two series. See
[metrics](https://sexpgpu.041.io/docs/metrics.md#runtime-metadata).

## Errors

| code | when |
|---|---|
| `E-CONTRACT-013` | two passes share a name |
| `E-CONTRACT-014` | an `evaluate` or a `curriculum` in a file with no pass, so nothing would run it |

Related: [curriculum](https://sexpgpu.041.io/docs/curriculum.md), [the status file](https://sexpgpu.041.io/docs/status.md).

---

SexpGPU documentation. Every page: https://sexpgpu.041.io/llms.txt