How does a small model learn chess moves?

1. Represent a move with three tokens

A ply is one player's move; a token is an integer ID. Both input families use IDs 0-63 for squares a1-h8. Each move becomes a source-square token, destination-square token and promotion token; 64 means no promotion. These three move targets keep the same meaning across the two input families.

These encodings do not assert that a move is legal in an arbitrary position. Castling and promotion require the appropriate board state. For the historical full-history format through CM-5, the sequence after 1. e4 e5 is [69, 12, 28, 64, 52, 36, 64]. BOS (69) starts this sequence. EOS (70) marks an observed game ending; PAD (71) is for batching. That format allocates 128 vocabulary IDs, including reserved IDs. CM-7 instead adds these same move targets to a 120-slot state prefix and allocates 512 vocabulary IDs. Tokens are sequence elements; vocabulary size is the number of possible IDs, not the number of tokens in an input.

Inspect the historical move tokenizer · Historical data preparation · Step through real games

2. Learn the next token, then assemble a move

The training objective is causal next-token prediction. At each position, the model sees the preceding tokens and predicts the next one. Causal attention prevents looking at later tokens. During training, earlier target tokens are the recorded training targets: this is teacher forcing.

A move needs three conditional predictions. The destination depends on the input prefix and source; promotion depends on that prefix, source and destination. There are no thought tokens or reasoning traces. CM-7 includes book-supported and exact WDL-preserving move targets; this is move supervision, not a search trace. Through CM-5, training uses complete histories. CM-6 and CM-7 instead train only the three move targets from a fixed state prefix; the input-family diagrams explain the distinction.

See the architecture diagram · Inspect the current objective and model · Read the current training experiment

3. Separate predictions from legal play

Raw probabilities are not a chess rules engine. The live scorer enumerates legal moves from the authoritative board and ranks their three-token joint probabilities. This policy turns model scores into a legal action. A separate bounded helper first searches for a forced mate; if proven, it bypasses the model. The controller separately checks freshness and legality before submission.

A KV cache reuses attention keys and values for the same input prefix while scoring candidate continuations. It accelerates inference; it is not a search tree or a reasoning trace.

See the exact branch ordering · Inspect legal scoring · Inspect the mate proof search

4. Ask what the measurements establish

Start with CM-7: 21 wins, 19 draws, 58 losses and two unfinished games. This is the whole playing system, including its mate helper, on a recorded Sunfish suite. Inspect the games, then compare protocols before comparing scores.

The historical foundation report compares identical validation samples and a frozen 100-game-per-model suite. It preserves an unfinished game and separates the evaluation helper from the later production helper.

Validation negative log-likelihood measures prediction on held-out tokens; lower is better under the same protocol. It is not an Elo rating. Teacher-forced legality diagnostics are also different from free-running game play.

The initial Sunfish match contains 20 games, including every loss. All three wins included helper overrides. An ablation would compare with the helper removed; none was run, so these games cannot establish whether the helper was necessary for those wins.

Compare the three evaluation layers · Read the match protocol and caveats · Download all 20 PGNs

5. Inspect first, reproduce with the boundaries in view

Start with a run's configuration, dataset provenance and checkpoint hash. Then inspect the matching source snapshot and check which weights and datasets are actually available for that version. Verify checksums before use. Original code is Apache-2.0; Lichess data are CC0 and dependencies retain their licenses.

Historical private paths are redacted, the pilot's exact source is unavailable, and optimizer state is excluded. This is not a one-command, bit-identical replay. Those gaps are documented rather than filled with invented code.

Follow the end-to-end source map · Check current reproduction artifacts and gaps · Read licensing