EXP-3 / completed / training

EXP-3: Learning full game histories

Scaling the same model to the retained monthly corpus will improve held-out next-token prediction within a fixed compute budget.

Non-reasoning next-move model: complete game history from the standard start, three-token causal objective, no thought or search supervision. Follow data → tokenizer → training → harness → bot bridge.

Baseline

Random initialization validation NLL 5.029830; pilot is contextual, not a comparable validation set.

Data

684,367 retained training games; 155,948,725 prediction targets per pass. Validation: 7,000 games / 1,595,704 targets. 305,193,023 repeated target exposures (1.957 target-pass equivalents), not unique data.

Configuration

Exact recorded configuration

{
  "accumulation": 2,
  "dataset_fingerprint": "e59e843f726f5ebcb54cd353f76a4e62c216829fa5085b830f9a534a9e9b441f",
  "epochs_schedule": 2,
  "lr": 0.0003,
  "max_updates": 41854,
  "microbatch": 8,
  "model": {
    "attn_implementation": "sdpa",
    "attn_pdrop": 0,
    "bos_token_id": 69,
    "embd_pdrop": 0,
    "eos_token_id": 70,
    "n_ctx": 1024,
    "n_embd": 768,
    "n_head": 12,
    "n_layer": 12,
    "n_positions": 1024,
    "pad_token_id": 71,
    "resid_pdrop": 0,
    "use_cache": false,
    "vocab_size": 128
  },
  "optimizer": "FP32 fused AdamW betas .9/.95 matrix wd .1 clip1",
  "parameters": 85940736,
  "precision": "FP32 weights/gradients/optimizer; BF16 autocast SDPA",
  "sampling": "seeded length buckets, disjoint full games; right pad; zero-target dummy rank on partial global batches",
  "schedule_steps": 41854,
  "seed": 2026,
  "tokenizer": {
    "sha256": "0db7085edec22885bd8c51aac4d6867afc04203ba913ef561be97213df019029",
    "version": "chess-square-triple-v1"
  },
  "torch": "2.9.0+cu128",
  "training_code": {
    "train_chess.py": "8147238e3684c68343c80a365d3c13f6d061e921b50fe9bcb28d75a18423b279",
    "training_core.py": "140022ab400e6ef5f56fe4c3a1df28834afdc7f90e39b011968418a93795b856",
    "training_deadline.py": "41aa6a01d17823f24d1990d181b5402c16aed96848569b30210d5bffc6d2e8dd",
    "verified_training_data.py": "ad3952fe45d8721d05dbe80d6f077e69a4851a3fae7ee5114def2e33ccc57037"
  },
  "warmup_steps": 2093,
  "world_size": 2
}

Compute

Two RTX 4090 GPUs. 41,854 updates in 86.01 minutes total job wall time; 63,058 useful targets/s during optimizer work.

Results

Recorded metrics — protocols differ by experiment
Metric	Value
Best validation NLL	0.6058687
Validation games	7,000
Teacher-forced triple legality (128-game subset)	0.6463698
Raw illegal-token probability	0.009694815

Validation learning curve (9 checkpoints)

Update	Validation NLL
5300	0.889934
10600	0.776929
15900	0.717477
21200	0.679090
21387	0.676202
26500	0.651721
31800	0.630323
37100	0.612812
41854	0.605869

What this does not establish

Test set and free-running match strength unmeasured in this run.
Teacher-forced legality uses 128 fixed validation games, not free-running rollout.
Full BF16 DDP exact resume equivalence remains untested; this run did not resume.
Single seed/configuration; not a hyperparameter optimum or production-readiness claim.
Same-month splits share players/opening positions; exact full-game ID/sequence duplicates are grouped.
Promotion accuracy is dominated by no-promotion tokens; do not interpret it as promotion strength.
The later Sunfish match uses a legal-move filter plus a mate helper and is a separate evaluation.

Provenance

Browse the recorded source snapshot →

Checkpoint SHA-256 6133648b2926b38542ee3ee1251a1ba111d61da9a8ea69d1c5f0559b09933643

Download experiment record (JSON)

Evidence: completion-summary.json

Original artifact SHA-256 2d1f69d863327fcfa150ec9e0e30409a8ffe277ce5b4c8dfd4bd85045ac94747

Selected public fields; private paths and process/account details excluded.

{
  "best_validation_nll": 0.6058686751172994,
  "final_validation": {
    "games": 7000,
    "legality_games": 128,
    "legality_subset": "N evenly spaced game indices across immutable validation shard order; N=None means all",
    "legality_targets": 28556,
    "nll": 0.6058686751172994,
    "phase_accuracy": {
      "destination": 0.7693100791588615,
      "eos": 0.111,
      "promotion": 0.999884811771104,
      "source": 0.5690109674300562
    },
    "phase_counts": {
      "destination": 529568,
      "eos": 7000,
      "promotion": 529568,
      "source": 529568
    },
    "protocol": "raw128 teacher-forced next-token logits; triple destination/promotion conditioned on gold prefix; not free-running",
    "raw_illegal_token_probability": 0.009694814930694023,
    "raw_reserved_probability": 0.0000015190403278775776,
    "seconds": 19.11114326200004,
    "teacher_forced_triple_legality": 0.6463697762769101,
    "valid_targets": 1595704
  },
  "test_evaluated": false,
  "total_wall_seconds_from_guardian_receipt": 5160.336741685867,
  "training_prediction_target_exposures": 305193023,
  "updates": 41854
}