EXP-18 / evidence study
CM-14: current, with clear boundaries
Which system produced the published 72%?
Keep in view. Single cohort; no matched comparator or confidence interval supplied for this older benchmark.
Published benchmark
100 games; mate-only assistance. Do not substitute this result for the newer 72.5% matched reference in CM-15 evaluation.
| Cohort | Wins | Draws | Losses | Unfinished | Score / bounds |
|---|---|---|---|---|---|
| CM-14 u210395 | 68 | 8 | 24 | 0 | 72% |
Model and interfaces
Current release identity is preserved; this presentation does not modify weights or serving configuration.
| Interface | Recorded configuration |
|---|---|
| Initialization | Fresh Muon/AdamW lineage; not CM-13 continuation |
| Benchmark | Mate-only |
| Live bot | Separate 97% endgame conversion enabled |
| Browser | Unassisted |
Download the study’s recorded values. Exact source records, hashes and protocol details remain below. Selectors inspect recorded observations; no missing points are interpolated.
Full experiment record, provenance and original links
Study source files and SHA-256 provenance
Original narrative and source links are preserved in the full record below. A linked release inventory may document identity without supplying every study value.
- cm14/release.json
d3dfc2815162b842dc6ec9e39988825dc8665526c5825316025377162758fc0a
EXP-18 / completed / CM-14 / continuation
EXP-18: CM-14 continuation and 100-game Sunfish evaluation
Update 210,395; promoted by user request after the 100-game Sunfish evaluation.
Evaluation
| Sunfish benchmark | Wins / draws / losses | Score |
|---|---|---|
| CM-14 continuation, 100 games | 68 / 8 / 24 | 72% |
Draws count as half a point. This is a development benchmark, not a general Elo estimate or unassisted browser strength. It does not establish a statistically significant advantage over another model.
The evaluated checkpoint used the matched Stockfish 19 mate-only helper protocol. The live bot retains its separately enabled 97% endgame conversion, which was not part of this evaluation. Browser play uses neither helper nor endgame conversion. Keep these configurations separate when interpreting results.
Model and lineage
The 86,966,016-parameter transformer reads clean, complete move history: one token per UCI move, including promotions. It has 12 layers, 12 heads, width 768, 512 context positions and 1,975 vocabulary entries. Move logits and fixed-White win/draw/loss logits are normalized separately. Outcome labels are not supplied as input, and the WDL diagnostic does not choose the browser move.
CM-14 began from fresh random initialization with a hybrid Muon/AdamW optimizer. This promoted checkpoint continues that CM-14 run; it is not a continuation of CM-13 weights.
Exact identity
Source SHA-256 31451d0d425ef14e020b847a3d1fefcbceacd32b0e1123adeab5f62877c379c2; inference export SHA-256 4cee1bf258f6061843f6073cd21d65f8981153ae47108551df617cb3053eea72.
Release metadata · Browser validation and source · Historical CM-13 evaluation