LittlePlyModel + harness

EXP-19 / evidence study

CM-15: inspect the forecast

Does predicting the eventual result mean stronger play?

Keep in view. Initial completed run only, not today’s stopped continuation. Not promoted. CM-16 remains a proposal.

Playing comparisons

New matched CM-14 Sunfish reference 72.5%; difference −4 points, paired interval −15 to +7. Both use frozen mate checks AND 97% endgame conversion. Head-to-head: 24 paired games score 41.7%, interval 29.2–54.2%; extra drawn Black game excluded from interval.

Playing comparisons — complete recorded values
CohortWinsDrawsLossesUnfinishedScore / bounds
Sunfish64927068.5%
CM-14 head-to-head7711042%

Outcome prediction by retrospective third

7,945 nonterminal positions / 100 games, game-equal weighting and opening-pair bootstrap. Thirds use completed game length, not online-known labels. Overall Brier .468 vs .518 cross-fitted baseline; difference interval −.120 to +.020. Nine draws limit calibration.

Outcome prediction by retrospective third — complete recorded values
ThirdBrier ↓Log loss ↓Actual-result probability
Early0.6060.9841.1%
Middle0.4880.82153.0%
Late0.3050.54368.3%

Download the study’s recorded values. Exact source records, hashes and protocol details remain below. Selectors inspect recorded observations; no missing points are interpolated.

A recorded forecast, not a simulation

98.55% to win. An eventual loss.

F100-097 at ply 57 is a retrospectively selected confident mistake. The prediction concerns the eventual result, not objective position quality. No independent engine adjudication was performed.

All recorded prefixes, FENs and raw fixed-White probabilities for this game · Original 100-game PGN

Full experiment record, provenance and original links

CM-15 / historical, not promoted

Forecasts are evidence. Results decide.

Initial completed run, update 339,215. Evaluation began 4 November 2026 at 05:13:45 UTC. CM-14 remains live.

Exact identity and evidence

Checkpoint SHA-256 fae796c2f7339650a1650066e7ff889c8f84df2acf577b122a6f9e98dc5030d5. Frozen CM-14 inference SHA-256 4cee1bf258f6061843f6073cd21d65f8981153ae47108551df617cb3053eea72.

All 125 games passed replay and legality audits, with no gameplay errors, unfinished games or deadline misses. Sunfish latency: median 0.961 seconds, p95 0.980, maximum 1.001. A summary-only error in 71 head-to-head helper records was repaired from before/after decisions; no games were rerun.

Outcome query 1973 follows clean history. Raw predictions are fixed White; report metrics are bot-relative. No outcome labels appear in inputs. Terminal predictions are preserved in raw data but excluded from calibration metrics. All 8,045 positions were covered, including 100 terminal positions; 7,945 remain for metrics. Post-game diagnostic queries did not choose moves.

Identity and checksums. This initial run is separate from the subsequently stopped continuation; CM-16 now has a separately documented full-run launch. No CM-15 promotion or completed CM-16 playing result is claimed.