Text input limits small AI models in playing riichi mahjong well
Text Is the Bottleneck: A Small-Scale Replication of LLM Game Playing in Riichi Mahjong
Computation and Language
Summary
The authors studied how well smaller AI models can learn to play the complex game of riichi mahjong by copying moves from a strong AI teacher. They found that using board data directly helps these models agree with a top AI's decisions more than text-based large language models (LLMs). Simply making models bigger is less helpful than improving how they are trained on accurate move data. However, even models that agree very closely with the expert do not play noticeably better in actual games. The study clarifies limits and best practices for building efficient game-playing AI with smaller resources.
What this means in practice
- •For game ai developers: Improve small-scale game-playing AI by focusing on direct board input and teacher imitation instead of large language model text prompts.
- •For consumer gpu users: Run effective riichi mahjong AI training using limited GPU resources by adopting compact models with convolutional encoders.
Authors
Mingge Xu
Abstract
We replicate the riichi-mahjong track of "Can Large Language Models Master Complex Card Games?" (arXiv:2509.01328) under a drastically smaller budget: models of at most 2B parameters, one 16 GB consumer GPU, about five GPU-days. (1) With matched data and labels, text is the bottleneck: on the same 100k-decision training set, a 2.6M-parameter convolutional board encoder agrees with the strong AI Mortal on 80.6% of test decisions, above every text-prompt LLM we train (best 77.8%), and places better at the table (2.85 over 1,000 games vs. ~3.0 for the LLMs; paired difference at n=100 not significant). Scaled to 1.71M decisions, a 4.9M-parameter Transformer reaches 94.7% agreement, beats the 0.8B LLMs on identical seeds (-0.45 ranks, sign test p=0.005), and is statistically equivalent to parity against three Mortal champions (TOST within 0.10 ranks, p=6e-4); parity against a teacher-opponent is roughly the ceiling of pure imitation. (2) Training toward the teacher raises teacher agreement more than a 2.5x parameter increase does (+2.5 vs. +1.5 points); on human-record agreement both effects are +1.0, and neither is visible at the table. (3) Few-point agreement differences do not resolve into playing strength within a family: +2.5 agreement points coincide with a paired placement difference of +0.02 +/- 0.12 ranks (p=1.0), and +2.0 points at 2B with -0.06 (p=0.37) - both below the ~0.25-rank resolution of 100 paired games. We also document a paper-code mismatch in the original winner filter, reconstruct its two private engine patches, and show that a divide-and-conquer pipeline makes template-generated Japanese rationales 93% sentence-exact (43% without facts). Per Tenhou's terms we release no trained weights - only code, configurations, and evaluation protocols.