Fixing MCTS for simultaneous-move games with decoupled UCB

I built a Code World Model (LLM-synthesized deterministic simulator, à la DeepMind’s CWM approach) for a simultaneous-move space strategy game, and ran into a problem DeepMind’s own approach doesn’t solve: standard MCTS and Information-Set MCTS both assume a single active player per node. That assumption breaks the moment both players act at the same instant, which is the actual structure of markets, auctions, and most multi-agent systems — not just games.

The fix: decoupled UCB. Instead of one joint action-value table, each player keeps an independent UCB table over their own actions, and the joint action is the Cartesian product of both players’ argmax picks. In two-player zero-sum settings this converges toward a Nash equilibrium instead of an exploitable pure strategy.

A few results from testing this (CWM + SM-MCTS vs. a sequential-MCTS baseline using the same simulator, same time budget, same weights — only the tree structure differs):

• 850–150 win/loss across 1,000 games (85% win rate) from the algorithm change alone
• Used CMA-ES to tune 24 value-function weights via self-play against a growing opponent pool rather than hand-tuning them — this surfaced non-obvious findings, like planet count being nearly irrelevant in 2-player but dominant in 4-player, and map centrality flipping from asset to liability as player count increases
• The determinism of a code-based world model (vs. a learned/neural one) is what makes this debuggable at all — you can trace exactly which transition produced a bad decision

Full technical writeup (architecture, the intercept-solving math, collision detection, feature design, CMA-ES setup, and the tie-back to order-book/market microstructure) is here: https://jdsemrau.substack.com/p/a-self-improving-code-world-model

Curious whether others have run into the same sequential-MCTS-on-simultaneous-games trap, and what approaches you’ve used to get around it.