EXP-9 / completed / System evaluation
EXP-9: State-conditioned system evaluation
CM-6 completed all 100 games: 13 wins, 7 draws, 80 losses and no unfinished games. Score: 16.5%.
Research-only; not a promotion candidate. CM-6 was not promoted. See the model registry for current serving versions.
System-level results
CM-6 used structured state, a 512-class decoder, BF16 GPU inference and an adaptive seven-ply helper. Historical CM-5 used full-history input, FP32 inference and a frozen three-ply helper. This system-level comparison does not isolate model, input, training duration, data, precision or helper effects. No old checkpoint was rerun.
| System | W / D / L | Unfinished | Points / 100 scheduled |
|---|---|---|---|
| CM-6 + adaptive 1/3/5/7 | 13 / 7 / 80 | 0 | 16.5 |
| Historical CM-5 + frozen 3-ply | 27 / 9 / 63 | 1 | 31.5–32.5 |
CM-6 scored 15–16 points lower on this selected suite, depending on the unresolved historical game. These outcome bounds are not confidence intervals or Elo. EXP-9 is deliberately separate from the matched CM-3/4/5 benchmark.
Protocol and diagnostics
Same 50 opening positions, paired colors and seeds; Sunfish e396948acbd80cdcbad3dccfb07e878fba9df8f4, 1,000 requested nodes (may overshoot), 256-ply cap including book autoplay and symmetric draw claims. Authoritative board history remains outside the model.
The Rust helper searches 1/3/5/7 plies with 50,000 nodes, a 250 ms budget and four shared workers. Its binary SHA-256 is c54530efd5a4e5f2b22e17290613f4ac89a2853906e5c1fa6ea2ed5d3d7881ca. All 21 overrides had verified all-defenses proofs. Of 3,195 model turns, the other 3,174 queried the neural model. Raw greedy triples were legal on 97.5425% of those queries; mean unmasked legal joint probability was 94.5230%. Actual policy moves used legal joint argmax. Book autoplay and helper overrides are excluded from raw-model denominators.
Helper outcomes: 21 proven, 1,313 completed depth-bound searches and 1,861 unknown (721 deadline-limited, 1,140 node-limited). Unknown does not mean no mate exists. Helper p95 was 182.0 ms; full model-turn p95 was 242.6 ms and maximum 420.0 ms. The helper budget does not bound the entire turn.
Audit and recovery
All 100 PGNs, outcomes, final boards, metrics and mate proofs passed audit. An audit ordering assumption failed after seven games; a reviewed multiset comparison correction preserved missing/duplicate-defense rejection, with 52 negative mutations tested. The original seven games were preserved and only the remaining 93 were run. No game was replayed or omitted. There were zero game execution/cleanup errors. Completion was approximately 2026-10-29 19:24:07 UTC.
Inspect the evidence
Replay all 100 games with CM-6 / EXP-9 filters · Download combined PGN · Audited metrics and hashes · Training and input structure
Combined PGN SHA-256: e46c0ef3929dc9a6ee9a11b6a8f7fb9859d65def8e6038494697fca83655c62a. Per-game start timestamps were not recorded; the archive marks them unknown rather than substituting upload or run-completion times.
What this experiment validates
CM-6 is research-only and is not a promotion candidate. Its purpose is to validate an end-to-end pipeline for training and inference from FEN-derived current board/state plus optional recent history, without requiring the complete game history from the standard start.
FEN describes the current board and state; it is not an incomplete board position. The available past history can be incomplete, and repetition history can be unknown. Optional recent moves do not reconstruct the entire game.
The completed one-hour training run demonstrates that this input and target pipeline can learn. Legal game play tests the integrated model, authoritative legality handling and external mate helper; it does not establish the raw model’s legality or playing strength by itself. Limited training duration is context, not causal proof of why a particular score occurs. No duration ablation or controlled input-only comparison was performed.
EXP-9 uses an adaptive 1/3/5/7-ply helper, unlike the frozen three-ply helper in EXP-6 and EXP-7. Any comparison is system-level. CM-6 remained a research checkpoint; subsequent CM-7 adoption is documented separately.