EXP-19 / evidence study
CM-15: inspect the forecast
Does predicting the eventual result mean stronger play?
Keep in view. Initial completed run only, not today’s stopped continuation. Not promoted. CM-16 remains a proposal.
Playing comparisons
New matched CM-14 Sunfish reference 72.5%; difference −4 points, paired interval −15 to +7. Both use frozen mate checks AND 97% endgame conversion. Head-to-head: 24 paired games score 41.7%, interval 29.2–54.2%; extra drawn Black game excluded from interval.
| Cohort | Wins | Draws | Losses | Unfinished | Score / bounds |
|---|---|---|---|---|---|
| Sunfish | 64 | 9 | 27 | 0 | 68.5% |
| CM-14 head-to-head | 7 | 7 | 11 | 0 | 42% |
Outcome prediction by retrospective third
7,945 nonterminal positions / 100 games, game-equal weighting and opening-pair bootstrap. Thirds use completed game length, not online-known labels. Overall Brier .468 vs .518 cross-fitted baseline; difference interval −.120 to +.020. Nine draws limit calibration.
| Third | Brier ↓ | Log loss ↓ | Actual-result probability |
|---|---|---|---|
| Early | 0.606 | 0.98 | 41.1% |
| Middle | 0.488 | 0.821 | 53.0% |
| Late | 0.305 | 0.543 | 68.3% |
Download the study’s recorded values. Exact source records, hashes and protocol details remain below. Selectors inspect recorded observations; no missing points are interpolated.
A recorded forecast, not a simulation
98.55% to win. An eventual loss.
F100-097 at ply 57 is a retrospectively selected confident mistake. The prediction concerns the eventual result, not objective position quality. No independent engine adjudication was performed.
Inspect the exact chronological input
These are recorded UCI moves, not reasoning or a generated explanation. The outcome query token 1973 is appended after this clean prefix; policy inference does not use that query.
All recorded prefixes, FENs and raw fixed-White probabilities for this game · Original 100-game PGN
Full experiment record, provenance and original links
CM-15 / historical, not promoted
Forecasts are evidence. Results decide.
Initial completed run, update 339,215. Evaluation began 4 November 2026 at 05:13:45 UTC. CM-14 remains live.
Exact identity and evidence
Checkpoint SHA-256 fae796c2f7339650a1650066e7ff889c8f84df2acf577b122a6f9e98dc5030d5. Frozen CM-14 inference SHA-256 4cee1bf258f6061843f6073cd21d65f8981153ae47108551df617cb3053eea72.
All 125 games passed replay and legality audits, with no gameplay errors, unfinished games or deadline misses. Sunfish latency: median 0.961 seconds, p95 0.980, maximum 1.001. A summary-only error in 71 head-to-head helper records was repaired from before/after decisions; no games were rerun.
Outcome query 1973 follows clean history. Raw predictions are fixed White; report metrics are bot-relative. No outcome labels appear in inputs. Terminal predictions are preserved in raw data but excluded from calibration metrics. All 8,045 positions were covered, including 100 terminal positions; 7,945 remain for metrics. Post-game diagnostic queries did not choose moves.
- SUNFISH-RESULTS.json
13eb3d9b563343ed29b8a79dadf635f82d993686f7d940b0ca5ddf128779f236 - head25-games/audit.json
7bf92a8346bf392aaafa02de85fb392c08db383a531f76456aff5d8525a7b0f2 - head25-games/games.pgn
7c60867c742fb9370624dc2c9f68a3a1070088aac2d75f299241bb631be7752c - sunfish-games/games.pgn
c635025bd41cb9f73df1f19a03369cf32dc19a6956b1a8a6337b5cc3524f8b77 - calibration/metrics.json
6c83999854780f5407e6d693ccca53810f84dc98ec65db0749218e23ffa6cdb4 - calibration/REPORT.md
7a53b03c979f84b792bb3867185d4067b25de49e2de96e42309a514e0f39dd99 - analysis/audit.json
bdbeca5093e223751346807b123be51bf184f444bd07096c06f262070421872f
Identity and checksums. This initial run is separate from the subsequently stopped continuation; CM-16 now has a separately documented full-run launch. No CM-15 promotion or completed CM-16 playing result is claimed.