Objective and evaluate

objective is what the optimizer minimizes. evaluate is what every evaluation pass reports. Both receive the model's prediction and the targets prepare returned.

(defun objective (prediction targets)
  (let ((loss (cross-entropy prediction targets :reduction :sum)))
    (metric "train/loss" (/ loss (numel targets))) ; what the summary shows
    loss))                                          ; what is optimized

(defun evaluate (prediction targets)
  (metric "eval/loss"
          (/ (cross-entropy prediction targets :reduction :sum)
             (numel targets))))

objective

• Returns one float scalar; anything else is E-OBJ-001, with the shape it did return.
• Several losses are one weighted sum; report the parts with metric.
• Its value is emitted every step as train/objective, averaged over the step's microbatches, whatever it is called and however it reduces.
• Written with three parameters, it also receives the extra keys of prepare's plist as metadata.
• Gradients are of the objective as written: a sum and a mean give different gradient scales. tap-gradient sees exactly that adjoint.

evaluate

• Returns nothing useful; it emits metrics with metric.
• Runs on every evaluation pass, after eval-prepare and the model, over that pass's loader.
• Each metric is averaged over the batches the pass reads, tagged with the pass name, and offered to the curriculum by name.
• Required when a pass exists; an evaluate with no pass is E-CONTRACT-014.

Two names the runtime reads

name	effect
train/loss	when objective reports it, the summary, the status file and the progress line show it instead of the objective, so a run optimizing a sum still prints the mean per token
eval/loss	when evaluate reports it, sweep's table shows it instead of the first metric the evaluation reported

metric, its metadata and what it becomes in each place are in metrics. A number about how the model is doing rather than how well is a diagnostic instead.

Related: the run file, sexpgpu/nn (cross-entropy).