Macros and symbols

defmacro receives unevaluated forms and returns a form, which is evaluated in the caller's lexical environment. Macros are unhygienic by default, as in Common Lisp: capture is allowed and sometimes the point. Gensyms protect introduced bindings; module identities protect a library's private helpers.

(defmacro twice (x)
  (once-only (x) `(+ ,x ,x)))          ; x is evaluated once

(defmacro/g! twice-explicit (x)
  `(let ((,g!value ,x)) (+ ,g!value ,g!value)))

The helpers

In the prelude unless noted:

form	behavior
`(gensym [prefix])`	a fresh uninterned symbol on every call, even across compilations in one process; a builtin
`(with-gensyms (name...) body...)`	binds each name to a fresh symbol while the body builds code
`(once-only (arg...) body...)`	in a macro body, generates bindings that evaluate each argument once, in order
`(defmacro/g! name lambda-list body...)`	each distinct `g!name` in the body becomes a fresh symbol per expansion
`(macroexpand-1 form [env])`	expands the outer macro call once; returns the form and an expansion flag
`(macroexpand form [env])`	expands the outer call until it is not a macro; the same two values
`(intern string [env])`	the symbol visible under that name in `env`, or one of this module
`(macrolet ((name ll body...)...) body...)`	local macros; expansion bodies see the enclosing environment

Expansion inspection does not evaluate the result or expand subforms. macroexpand defaults to its own lexical environment; &environment receives the caller's. Put (macroexpand-1 '(twice (f y))) in a file and sexpgpu eval prints the expansion.

Symbols and capture

A symbol belongs to its module, or is a fresh symbol from gensym. Equal spelling does not make two modules' private symbols equal.
eq and equal compare identity. symbol-name returns the printable name, which is not the identity. Gensyms print as #:name; that is not reader syntax.
An unbound ordinary symbol falls back to the prelude by name, so a module may define its own when without changing anyone else's.
A library macro's references to its own helpers keep the library's identities: the caller imports the macro, never its helpers.
Unquoted arguments keep their original identities and bindings.
Importing a symbol shares its identity, and because functions and variables share a namespace, a let of an imported function name captures it.

Intentional capture across a module boundary uses intern with the caller's environment:

(defmacro aif (test then else &environment caller)
  (let ((it (intern "it" caller)))
    `(let ((,it ,test)) (if ,it ,then ,else))))

(aif 7 (+ it 1) 0) ; 8

Generated declarations

Definitions register when their form executes. A macro can generate defun, defmacro, defmodel, defoptimizer, defvar, defknobs and defvariant, and another file can import the top-level names it produced.

(defmacro define-width (name width)
  `(defvariant ,name (width ,width)))

(define-width wide 512)
(defknobs (width 128))

Each form in a top-level progn resolves its symbols after the forms before it ran, exactly like separate top-level forms.
Variants, generated or not, come before the first defknobs; see knobs.
Local definitions stay local; imported names are not re-exported.
Duplicate definitions and conflicting imports fail at the second binding, E-EVAL-036.

Names in the model and the optimizer

Gensym bindings never name persistent things. A model bound with a gensym gets no path segment; a parameter declared with a gensym gets a stable param.N, and a gensym optimizer state a stable state.N, in declaration order. Give persistent parts ordinary names or (named "segment" ...); see models and optimizers.

Diagnostics through macros

Generated code keeps the macro call, the definition and the template origin. The notes survive into generated closures and show up when a later trace fails, so an error in expanded code still points at the call you wrote.

A worked example

A macro generates a model constructor, aif from above captures it on purpose, and a momentum optimizer step keeps its state under a gensym:

(defknobs (degree 3 "number of polynomial coefficients"))

(defmacro define-polynomial (name)
  `(defmodel ,name (degree &key (policy :high))
     (defparam coefficients (zeros [degree]) :numerics policy)
     (lambda (x)
       (let ((result (zeros-like x)))
         (dotimes (i degree result)
           (setq result
                 (+ result
                    (* (sum (slice coefficients 0 i (+ i 1))) (pow x i)))))))))

(define-polynomial polynomial)
(defvar model (aif degree (polynomial it) (error "degree is required")))

(defmacro momentum-step (p g rate)
  (with-gensyms (velocity next)
    `(progn
       (defstate ,velocity (zeros-like ,p))
       (let ((,next (+ (* 0.8 ,velocity) ,g)))
         (optimizer-update (- ,p (* ,rate ,next)) ',velocity ,next)))))