LittlePlyModel + harness

EXP-17 / evidence study

CM-13: the final snapshot

How did the later snapshot fare on matched openings?

Keep in view. Sunfish evidence, not general Elo or proof that duration alone caused improvement.

Matched 50 opening pairs

Gain +14.5 points; paired opening-cluster bootstrap 95% interval +1.5 to +27 points, 20,000 resamples. All 200 games audited.

Matched 50 opening pairs — complete recorded values
CohortWinsDrawsLossesUnfinishedScore / bounds
Final u27939363928067.5%
Fresh eight-hour reference471241053%

Protocol boundaries

Identical helper and timing for both evaluated snapshots.

Protocol boundaries — complete recorded values
InterfaceAssistance
Evaluated systemStockfish 19 mate-only
BrowserUnassisted
Excluded from evaluationValue search and 97% takeover

Download the study’s recorded values. Exact source records, hashes and protocol details remain below. Selectors inspect recorded observations; no missing points are interpolated.

Full experiment record, provenance and original links
Study source files and SHA-256 provenance

Original narrative and source links are preserved in the full record below. A linked release inventory may document identity without supplying every study value.

EXP-17 / completed / CM-13 / final snapshot

EXP-17: CM-13 final snapshot and matched 100-game evaluation

Update 279,393 replaces the earlier eight-hour snapshot; the earlier record remains intact.

Final snapshot versus eight-hour reference

Matched Sunfish benchmarkWins / draws / lossesScore
CM-13 final, 100 games63 / 9 / 2867.5%
CM-13 eight-hour reference, 100 games47 / 12 / 4153%

Observed gain: 14.5 percentage points. Paired opening-cluster bootstrap 95% interval: +1.5 to +27 percentage points (50 paired openings, 20,000 resamples). Draws count as half a point. This supports improvement on this Sunfish benchmark; it is not general Elo or a browser-strength estimate.

Both cohorts used identical Stockfish 19 mate-only assistance and timing, with experimental outcome search off. Sunfish used 1,000 requested nodes; games used 300+2 clocks and a one-second minimum opportunity. The helper used two engines, two threads and 64 MiB per engine. The later 97% endgame takeover was not part of this evaluation. Browser play has neither this helper nor an endgame takeover.

All 200 games passed identity, replay and per-ply WDL audits, with no illegal moves, errors, forfeits, unfinished games or deadline misses. Opening/training exposure was not independently excluded. These are matched system results under a specific protocol, not an isolated causal test of training duration.

Training and input

Fresh random initialization with AdamW; clean game history, one token per complete UCI move (one ply), including promotions. The 86,966,016-parameter transformer has 12 layers, 12 heads and width 768, with a 512-token context and 1,975 vocabulary entries. No masking and no outcome labels in inputs. Move outputs and fixed-White WIN/DRAW/LOSS outputs are separately normalized from the shared transformer and tied output matrix.

This is the final snapshot of the same fresh CM-13 run, not a new CM number or a continuation from CM-10. The eight-hour experiment preserves the earlier comparison against CM-10; its 54% result and this newly matched 53% reference are different evaluations.

Exact identity and serving boundary

Source SHA-256 2db161c6c8396388565f414123ccbe413deec2d0a85faee530ea8b5913a16ae4; inference export SHA-256 745b457f7751d912b878fd5242243e058ad2da2fe3eae1b002fbc40d2ffd0cbc. Lichess release: 745b457f7751-35bdb54f6540. The browser uses an FP32 ONNX conversion of those weights, with its own hash and parity evidence.

Evaluation and identity metadata · Browser implementation and validation · Previous snapshot identity. Promotion does not claim that unassisted browser play reproduces the Stockfish-assisted benchmark.

Related reading

Context for interpreting this experiment, not a claim that it implements these methods: Win-Draw-Loss evaluation; Emergent World Models and Latent Variable Estimation in Chess-Playing Language Models; Evidence of Learned Look-Ahead in a Chess-Playing Neural Network.