EXP-12 / completed / Evaluation / fixed suite
EXP-12: CM-7 final system evaluation
Two further games reached the 256-ply cap and remain unfinished. This is a system-level result on a selected fixed suite, not Elo or a general strength claim.
| Evaluation | W / D / L | Unfinished | Download / replay |
|---|---|---|---|
| Pre-pass diagnostic (10) | 4 / 3 / 3 | 0 | PGN · Replay |
| Update 1,500 diagnostic (10) | 4 / 1 / 5 | 0 | PGN · Replay |
| Selected CM-7 (100) | 21 / 19 / 58 | 2 | PGN · Replay |
| Historical CM-6 (100) | 13 / 7 / 80 | 0 | EXP-9 |
The unfinished games are opening-39-white and opening-49-white. CM-7 earned 30.5 points in 98 finished games: outcome bounds are 30.5–32.5 points per 100 scheduled, versus CM-6’s 16.5. The paired improvement bounds are 14–16 points on this suite. These are unresolved-outcome bounds, not confidence intervals.
The selected checkpoint’s matching ten-game subset is 3W / 4D / 2L / 1 unfinished. Both actual mini suites overlap the full suite and must not be pooled as independent evidence. The small diagnostics do not establish monotonic theory-stage improvement. All 120 actual games are published, including losses and unfinished games.
Protocol and policy
50 frozen openings, paired colors/seeds; Sunfish requests 1,000 nodes (overshoot retained); 256-ply cap; legal joint decoding with raw 512-class diagnostics. Same adaptive iterative 1/3/5/7-ply mate helper as CM-6, bounded by 250 ms, 50,000 nodes and four workers. Book autoplay is excluded from model diagnostics. Historical CM-5 used a frozen three-ply helper, so CM-5 comparisons change the system and do not isolate weights.
3,340 model queries; 99.2216% raw-greedy legal triples; 48 proven helper overrides. Helper latency mean 104.76 ms / p95 181.84 ms / max 184.27 ms. Entire model turns mean 154.95 ms / p95 240.65 ms / max 560.73 ms. The helper budget is not a whole-turn guarantee. Every PGN, move and mate proof passed the final audit; no execution/cleanup failures.
Interpretation
CM-7 is being adopted for the state-based input structure and its support for standalone-position/endgame training. This evaluation does not demonstrate clear superiority over the previous CM-5 production system, isolate the theory stage from its preceding curriculum, or establish broad generalization. Training and evaluation history remain independently inspectable.
Individual game-start timestamps were not recorded and remain unknown in the archive. The mini audit’s default scheduled-count label was corrected from 100 to 10 without changing outcomes, PGNs or runner behavior.
Inspect and reproduce
Training counts and checkpoint lineage · Evaluation evidence and hashes · Download allowlisted source · Original/rendered source hashes · Execution-pin matches
State codec and model · Data preparation · First-stage trainer · Follow-up trainer · Evaluation harness · Separate GPL Rust helper
Reproduction instructions and availability boundaries. Public source replaces host-specific paths with /work placeholders; both hashes are recorded. Historical snapshots remain unchanged.