TRON: Targeted Rule-Verifiable Online Environments for Visual Reasoning RL
2026-06-01 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors created TRON, a system that generates unlimited new visual reasoning questions on demand for training reinforcement learning models. Instead of relying on fixed datasets, TRON produces fresh images, questions, and verifiable answers tailored to different skill levels. This approach helps train models more effectively across various visual reasoning tasks without needing extra data collection. They also analyze TRON’s reliability and diversity, showing consistent improvements on multiple testing benchmarks.
reinforcement learningvisual reasoningtraining signalsdata generationcurriculum learninglatent visual stateverifiable answersbenchmarkingmultimodal reasoningmodel training
Authors
Tianze Yang, Yucheng Shi, Ruitong Sun, Jingyuan Huang, Ninghao Liu, Jin Sun
Abstract
Reinforcement learning (RL) for visual reasoning needs scalable, verifiable, and controllable training signals. Existing visual RL post-training trains on static curated datasets, with fixed image-question-answer samples bounded by their collection budget. In this work, we introduce TRON (Targeted, Rule-verifiable Online eNvironments), an online environment substrate: a training rollout is generated on demand by a controllable generator-verifier program that samples a fresh latent visual state, renders an image, asks a question, and exactly verifies the answer. A single run can therefore draw an unbounded stream of fresh instances at the difficulty level required by the current curriculum. The current TRON suite contains 520 environments organized into five ability buckets (spatial, mathematical, diagram, pattern/logic, and counting); the same substrate supports both a single full model trained on all buckets and per-bucket ability-specialist models, with no additional data collection. We also introduce a substrate analysis covering generation reliability, instance and level diversity, cross-environment near-duplicates, and base-model pass rate by difficulty level. RL post-training with METHOD consistently improves performance on ten external multimodal reasoning benchmarks across Qwen3-VL-4B, Qwen2.5-VL-7B, and MiMo-VL-7B-SFT.