# Objective and evaluate

`objective` is what the optimizer minimizes. `evaluate` is what every
evaluation pass reports. Both receive the model's prediction and the
targets `prepare` returned.

```lisp
(defun objective (prediction targets)
  (let ((loss (cross-entropy prediction targets :reduction :sum)))
    (metric "train/loss" (/ loss (numel targets))) ; what the summary shows
    loss))                                          ; what is optimized

(defun evaluate (prediction targets)
  (metric "eval/loss"
          (/ (cross-entropy prediction targets :reduction :sum)
             (numel targets))))
```

## objective

- Returns one float scalar; anything else is `E-OBJ-001`, with the shape it
  did return.
- Several losses are one weighted sum; report the parts with `metric`.
- Its value is emitted every step as `train/objective`, averaged over the
  step's microbatches, whatever it is called and however it reduces.
- Written with three parameters, it also receives the extra keys of
  `prepare`'s plist as `metadata`.
- Gradients are of the objective as written: a sum and a mean give
  different gradient scales. [`tap-gradient`](https://sexpgpu.041.io/docs/writing-diagnostics.md#gradient-diagnostics)
  sees exactly that adjoint.

## evaluate

- Returns nothing useful; it emits metrics with `metric`.
- Runs on every [evaluation pass](https://sexpgpu.041.io/docs/evaluation.md), after `eval-prepare` and
  the model, over that pass's loader.
- Each metric is averaged over the batches the pass reads, tagged with the
  pass name, and offered to the [curriculum](https://sexpgpu.041.io/docs/curriculum.md) by name.
- Required when a pass exists; an `evaluate` with no pass is
  `E-CONTRACT-014`.

## Two names the runtime reads

| name | effect |
|---|---|
| `train/loss` | when `objective` reports it, the summary, the status file and the progress line show it instead of the objective, so a run optimizing a sum still prints the mean per token |
| `eval/loss` | when `evaluate` reports it, `sweep`'s table shows it instead of the first metric the evaluation reported |

`metric`, its metadata and what it becomes in each place are in
[metrics](https://sexpgpu.041.io/docs/metrics.md#reporting-from-the-experiment). A number about how the
model is doing rather than how well is a
[diagnostic](https://sexpgpu.041.io/docs/diagnostics.md) instead.

Related: [the run file](https://sexpgpu.041.io/docs/run-file.md), [sexpgpu/nn](https://sexpgpu.041.io/docs/nn.md) (`cross-entropy`).

---

SexpGPU documentation. Every page: https://sexpgpu.041.io/llms.txt