Vs MuZero

The Comparison

What MuZero Adds Over AlphaZero

MuZero is AlphaZero plus a learned world model. AlphaZero needed the rules engine (chess.Board, GoBoard) at every MCTS node — every simulated child is a real legal position. MuZero removes that requirement by training three networks that together simulate the environment:

real observation o_t
        │
        ▼
   h_θ(o_t) = s⁰_t        latent state at the root
        │
        ▼   ┌─────────────────────────────────────────────────────┐
   MCTS    │  recursive descent through the LEARNED dynamics g_θ │
            │   g_θ(s_t^k, a_{t+1}) = (s_t^{k+1}, r_t^{k+1})      │
            │   prediction f_θ(s_t^k) = (π_t^k, v_t^k)            │
            └─────────────────────────────────────────────────────┘
        │
        ▼
   visit-count π over root actions → move played

The dynamics function g_θ is trained to predict what happens only on the trajectories MCTS visits, not the full state space. That means MuZero never has to learn “what’s a legal move in chess” in general — only enough to evaluate the moves its own search considers. That’s much less than learning chess rules ab initio, but it’s enough to match AlphaZero’s strength on chess + shogi + Go and also play Atari (where rules are unknown).

What This Project Has That MuZero Doesn’t Care About

We assume the rules. wm_chess.board.encode_board reads python-chess’s legality directly, MCTS only descends legal moves, game termination is determined by board.is_game_over(). That’s a massive assumption MuZero deliberately drops — but for our problem (can a small ResNet distilled from Stockfish play strong chess?) it’s the right assumption. Re-learning chess rules from scratch would be ~10⁶ extra positions of training signal that we don’t have, on top of the actual hard problem (learning to play well).

A cleaner way to say it: the world-model-learning question is genuinely orthogonal to the question this project asks. AlphaZero and MuZero hit the same Elo on chess; the difference is what they needed to see to get there.

Why Our Network Is “AlphaZero” Not “MuZero”

Three structural reasons:

1. We feed in real board state, not pixel observations. The 19-plane encoding (encode_board) is a hand-crafted representation of piece positions, castling rights, en-passant, and the side to move. MuZero’s representation network h_θ is learned from raw inputs (e.g. screen pixels for Atari). For board games MuZero also uses the same hand-crafted planes — so on chess specifically MuZero’s representation network is doing little work.
2. MCTS calls the real env. Every descent in our MCTS calls board.push(move) on a python-chess Board. MuZero would call g_θ(s, a) on the latent. We benefit from the real env being exact and free; MuZero benefits from generalizing past known-env problems.
3. We have a teacher. Stockfish labels positions for us. MuZero has no teacher — it learns from MCTS visit counts on its own self-play games (same as AlphaZero). The teacher means we don’t need MuZero’s world model to deliver a strong move-distribution target — Stockfish already gives us one for free.

Where MuZero Would Help (and we ignored it)

MuZero’s auxiliary losses anticipated a lot of what KataGo later formalized. The reward-prediction head in g_θ is morally the same shape as KataGo’s score head. The recurrent latent unrolling forces the network to keep enough internal state to plan multi-step — which helps even with known-env board games because it forces the encoder to make planning-useful features explicit.

We don’t have any of that. Our network outputs a per-position policy and a single scalar value; that’s it. KataGo’s paper documented ~9× efficiency gains over vanilla AlphaZero from adding aux heads of exactly this flavor (see vs-Lc0 page). We’ve plumbed the architecture for KataGo-style aux heads on the Go side but haven’t enabled them in a real run yet.

Is “Distill from Stockfish” Actually MuZero-Adjacent?

Sort of. Both AlphaZero/MuZero ultimately train the value head on game outcomes (the z target) and the policy head on improved distributions (π from MCTS). When we use Stockfish multipv as the policy target, we’re using a much stronger π than MCTS would produce at the same network strength — Stockfish at depth 15 sees much further than 800 MCTS sims of our smallish ResNet. So our policy training signal is structurally similar but bandwidth-rich.

What we don’t get from Stockfish-distillation: experience with the network’s own search distribution. MCTS’s blind spots (where priors disagree with deep MCTS) are exactly the positions where self-play would help most, and exactly the positions Stockfish’s labels can’t directly inform. That’s the residual gap between distill ceiling (2,300-ish) and Lc0’s 3,600+. The self-play attempts documented on the what’s next page target that gap.

In One Sentence

MuZero kept AlphaZero’s MCTS-at-inference and learned the rules engine; we kept AlphaZero’s MCTS-at-inference and used the rules engine plus a Stockfish teacher. Both are valid moves; ours is the right one for “single GPU + want a strong chess net fast.”