# CM-14: 10-second uncertainty-search evaluation

## Result

**73 wins, 12 draws, 15 losses: 79.0% score over 100 Sunfish games.**
This includes one genuine clock loss. There were no capped or unfinished games.
The opening-pair bootstrap 95% interval for the overall score is **72.5%–85.5%**.

- White:35 W/5 D/10 L,75.0%.
- Black:38 W/7 D/5 L,83.0%.
- Natural-only diagnostic:73 W/12 D/14 L over 99 games,79.8%. This excludes the clock loss and is not the headline playing-strength score.

All 100 games were independently replayed from immutable receipts. Legal moves, exact final positions, PGN moves/results and unique identities passed. A clock-forfeit PGN correctly records a loss even though the final board is nonterminal.

## Comparisons and uncertainty

The model checkpoint, openings and colors match the references. Historical PGNs were checked against the actual opening histories, results and final positions.

- Historical mate-helper system:72.0%. New score difference **+7.0 percentage points**, paired 95% interval **−3.5 to+18.0 points**.
- Historical mate-helper plus endgame-takeover system:72.5%. Difference **+6.5 points**, paired 95% interval **−4.0 to+17.0 points**.
- Preserved 3-second decoder subset:48 games/24 opening pairs. On those same games,10-second search scored 79.17% versus 68.75%, a **+10.42-point** difference; paired 95% interval **−5.21 to+25.0 points**.

**Every improvement interval crosses zero.** These results are promising but do not establish a reliable strength gain. The historical systems used helpers that this decoder did not. The 3-second comparison is an incomplete, nonrandom sequential subset, not a completed 100-game control. Reused openings, different compute and noncontemporaneous host load further limit causal conclusions.

Intervals are percentile bootstraps with 10,000 resamples of opening pairs, seed 20261106. They are conditional on this fixed suite and checkpoint, exploratory, and not adjusted for multiple comparisons. They are not universal Elo estimates.

## What extra thinking bought

For 3,739 recorded CM-14 decisions:
- Median decision time 3.10 s;95 th percentile 8.86 s; maximum 9.92 s.
- Uncertainty triggered 2,024 times. Search completed depth 3 on 1,906 decisions and depth 2 on 118;1,715 confident decisions stayed at the policy choice.
- Search changed the model’s top-policy move 1,162 times.
- 77,458 recorded model forwards; mean root-policy mass excluded by top 3 candidate selection was 14.93%.

On the matched 48 games, recorded CM-14 decision time rose from 2,887 s to 5,588 s, approximately **1.94×**. Mean recorded active game time rose from 61.8 s to 118.1 s. Model-decision 95 th-percentile time rose from 2.95 s to 8.81 s. Trajectories and move counts differ, so this is measured system cost, not a matched-node ablation.

The 10-second setting is a ceiling. Confident decisions and completed depth 3 trees stop early. In smoke testing, two uncertain positions completed depth 3 in about 2.75/2.81 s with 40 forwards each, peak memory about 1.066 GB and no deadline cutoffs.

## Clock-management risk

Game F100-047 was lost on time with CM-14 playing White. The 83 recorded White moves imply approximately **5.0597508 s remaining before the unrecorded forfeiting attempt**, reconstructed from 300+2 and saved decision durations. The last successfully recorded White decision took 9.7957021 s.

The timed-out attempted decision was not saved, so its duration, selected move and telemetry are unavailable. Do not mistake the previous 9.7957 s decision for the forfeiting attempt. Recorded timing/forward totals omit that final attempt.

The tested controller uses a fixed 10-second ceiling without adapting it to remaining clock time. **Clock-aware budgeting should be fixed and validated before considering this configuration for live use.** No live default was changed by this evaluation.

## Fixed protocol

Same CM-14 checkpoint; Sunfish 1,000 nodes, Hash 16;300+2 clocks;50 paired openings,100 games. No Stockfish helper or endgame takeover.

Search becomes eligible after three completed game plies, so the first eligible decision is game ply 4. All test openings already have 16 plies; consequently the early-opening gate was exercised by tests, not contrasted in these matches.

Search triggers when the top legal-move probability is below 0.60 or its lead over the second move is below 0.15. It uses the top 3 model-proposed legal moves per node, iterative deepening through depth 3, adversarial fixed-White W+0.5 D values and exact terminal rules. Only complete root iterations can replace the policy fallback. This is truncated candidate search, not full-width chess search.

## Files

- games.pgn: all 100 games.
- game-results.csv: compact game-by-game outcomes.
- results-summary.json: detailed scores, intervals, telemetry and comparison caveats.
- matched-3second-subset.csv: the 48 exact comparison identities and outcomes.
- audit.json: independent replay status and source identities.
- SHA256SUMS.txt: packaged-file checksums.

Raw decision journals and operational control files are intentionally excluded.
