Explicit curriculum training matches lLM methods without action masking
Anchor-and-Harvest Curriculum: A Fully Unmasked Non-LLM Curriculum for Craftax
Machine LearningArtificial Intelligence
Summary
This paper shows how a carefully designed training system called Anchor-and-Harvest Curriculum (AHC) can teach an AI to perform well in a complex game called Craftax-Full without using large language model (LLM) inference. The authors combined different learning techniques like adjusting difficulty and practicing original tasks without hiding any possible actions from the AI. Their method achieved performance close to systems that use much larger LLMs, showing that explicit, rule-based curriculums can be very effective. They also analyzed which parts of the game were still too hard and how constraints limit learning progress.
What this means in practice
- •For reinforcement learning engineers: Use AHC to train agents in complex domains without relying on large language models or action masking techniques.
- •For game ai developers: Apply rule-based curriculum methods to improve AI performance on intricate game environments while preserving full action sets.
Authors
Jiayu Zhu, Luyuan Li, Yuyuan Shi, Hanyi Dong
Abstract
We introduce Anchor-and-Harvest Curriculum (AHC), a domain-informed, rule-based training system for Craftax-Full. AHC combines achievement-oriented tasks, harvested initial-state progress, difficulty adaptation, bounded selection, and original-task practice. It uses no LLM inference and keeps all 43 actions available. At 2,005,401,600 PPO rollout transitions, three primary AHC runs obtain native-task return $46.70\pm0.57$ (sample SD), compared with $41.91\pm1.01$ for fresh native PPO-GTrXL with the same student architecture and evaluation protocol. Evaluation uses ten stochastic rollouts on each of 1024 fixed worlds per model. An additional cohort of three AHC runs reaches $46.08\pm1.23$. Two single-component ablations retain similar mean returns, while jointly freezing difficulty and removing inherited progress gives $45.14\pm2.79$. All-67-achievement profiles show that most of the return difference comes from Intermediate achievements, while the deepest-floor achievements remain unsolved. A Skill-Preflight diagnosis with a 14B teacher shows how prerequisite and floor-access constraints restrict practice. AHC and three 14B diagnostic configurations reach aggregate returns near the reported 80B DiCode mean of 45.88 and below the 235B mean of 48.55. These results show that an explicit curriculum without LLM inference can reach returns comparable to those reported for LLM-generated curricula on Craftax-Full, without action masking.