LittlePlyModel + harness

Chapter 05 / The experiments

What did training teach us?

Trace CM-14’s recipe and the limits of auxiliary objectives.

CM-14: learn policy and outcomes

CM-14 is an approximately 87-million-parameter transformer trained from scratch, not a prompted general-purpose model. It learned from chess move histories using a hybrid Muon/AdamW optimizer, followed by human-game continuation.

The useful interface is simple: a policy over moves and separate fixed-White WDL estimates. A legal rules layer and search then turn those signals into an agent.

CM-14’s recorded training and evaluation →

What an auxiliary objective asks

An auxiliary objective adds another learning task alongside the main one. CM-15 explored a future-move target. CM-16 explored future-move and board-state supervision alongside next-move and outcome learning.

The idea is plausible: learning structure might help the core task. But an extra target also spends finite computation and can change optimization behavior. More supervision is not automatically more useful progress per hour.

The experiment did not establish a replacement

CM-16 continued over multiple days, then dropped the extra objectives and continued with next move and WDL. It did not establish a stronger replacement for CM-14. The full record retains the checkpoints, periodic matches, interruptions and incomplete final assessment.

Hypothesis, not conclusion

Under a limited compute budget, auxiliary objectives may reduce throughput and divert effort from the predictions that matter most for play.

This is not an isolated causal result. Initialization, dataset, optimization and runtime differences also matter. A controlled study would hold those fixed and compare equal-time as well as equal-target budgets.

What to carry forward

Keep objectives tied to a measured use. Separate training experiments from harness improvements. Preserve unsuccessful runs instead of rewriting the story around the best score.

CM-16: timeline and evidence · CM-15 study · Reproduction boundaries