EXP-2 / completed / training

EXP-2: A short training pilot

A causal square-token model can learn next-move structure from a small full-game corpus.

Non-reasoning next-move model: complete game history from the standard start, three-token causal objective, no thought or search supervision. Follow data → tokenizer → training → harness → bot bridge.

Baseline

Initial random weights: validation NLL 5.027132.

Data

1,816 training games; 408,178 prediction targets per pass. Validation: 17 games / 3,659 targets. September 2026 Lichess pilot; 20-game test split not evaluated.

Configuration

Exact recorded configuration

{
  "accumulation": 2,
  "dataset_fingerprint": "a2b2709c431098c8230771629352c17ba747a884a7fae318bd44429a388aade7",
  "epochs_schedule": 20,
  "lr": 0.0003,
  "microbatch": 8,
  "model": {
    "attn_implementation": "sdpa",
    "attn_pdrop": 0,
    "bos_token_id": 69,
    "embd_pdrop": 0,
    "eos_token_id": 70,
    "n_ctx": 1024,
    "n_embd": 768,
    "n_head": 12,
    "n_layer": 12,
    "n_positions": 1024,
    "pad_token_id": 71,
    "resid_pdrop": 0,
    "use_cache": false,
    "vocab_size": 128
  },
  "optimizer": "FP32 fused AdamW betas .9/.95 matrix wd .1 clip1",
  "parameters": 85940736,
  "precision": "FP32 weights/gradients/optimizer; BF16 autocast SDPA",
  "sampling": "seeded length buckets, disjoint full games; right pad; zero-target dummy rank on partial global batches",
  "seed": 2026,
  "tokenizer": {
    "sha256": "0db7085edec22885bd8c51aac4d6867afc04203ba913ef561be97213df019029",
    "version": "chess-square-triple-v1"
  },
  "torch": "2.9.0+cu128",
  "training_code": {
    "train_chess.py": "9461ebe732075af809ab433b9c3269a21654228d1d940efccca081ec19b6534b",
    "training_core.py": "a4a778a52f2391ca4576e6d1329b9e44042b7b47467fc6286512ae51e4ed2d8d"
  },
  "world_size": 2
}

Compute

Two RTX 4090 GPUs; FP32 parameters and optimizer, BF16 autocast. Whole process 186.02 seconds.

Results

Recorded metrics — protocols differ by experiment
Metric	Value
Best validation NLL	1.510176
Best update	456
Final update	741

Validation learning curve (13 checkpoints)

Update	Validation NLL
57	2.062963
114	1.966790
171	1.837583
228	1.707228
285	1.608458
342	1.555289
399	1.521743
456	1.510176
513	1.519859
570	1.568135
627	1.661480
684	1.739081
741	1.900572

What this does not establish

Only 17 validation games; early stopped after five non-improving checks.
Teacher-forced raw-logit diagnostics are not free-running playing strength.
Pilot validation and full-month validation differ; their NLLs are not a controlled head-to-head comparison.
No test evaluation and no Elo claim.

Provenance

Browse the recorded source snapshot →

Checkpoint SHA-256 f64e1e69eff81a6757f421fdfafd31e77d5d0ab2d88258e73130de014fbc7f7b

Download experiment record (JSON)

Evidence: results.json

Original artifact SHA-256 a01c7c106aab7f6dde7d6d25a49a62252ab2a90f139a9d2b1739b8ff47809a1a

Selected public fields; private paths and process/account details excluded.

{
  "best_step": 456,
  "best_validation_nll": 1.5101759066898401,
  "test_evaluated": false
}