Continuous latent reasoning improves math problem solving accuracy
CALR: Continuous Anchored Latent Reasoning via Render-of-Thought Compression
Computer Vision and Pattern Recognition
Summary
Solving complex math problems can be slow because computers often explain their thinking step-by-step using words. This paper looks at ways to simplify and speed up that process by compressing the computer's reasoning into smaller hidden memory states, called latents. The authors found that linking these compressed states directly to problem-solving outcomes and making sure they hold meaningful information improves accuracy. They introduced a new method called CALR that does this linking better, leading to much higher success rates on math tests compared to previous methods.
What this means in practice
- •For ai software developers: Create math-solving AI systems that use compressed reasoning steps to improve speed and accuracy within limited computing resources.
- •For automated tutoring platform teams: Develop tutoring tools that better predict correct answers by improving intermediate reasoning representations for math problem solving.
Authors
Zhaoyang Wei, Bowen Jiang, Yanchao Hao, Wenchao Ding, Zheng Wei, Shaocheng Wu, Zhenjun Han, Jianbin Jiao
Abstract
Visual latent reasoning compresses rendered derivations into compact intermediate states, reducing textual reasoning overhead. Existing approaches differ in how they represent these states: continuous methods avoid vocabulary constraints, whereas discrete methods improve accuracy through quantization into a finite codebook. Our analysis of representative continuous and discrete systems identifies two functional requirements: answers must rely on latent states, and those states must carry valid, problem-specific reasoning. Continuous latents influence answers despite collapsed reasoning content, whereas discrete latents retain recoverable intermediate reasoning that answer prediction largely bypasses. To address these challenges, we propose Continuous Anchored Latent Reasoning (CALR), which connects latent formation with answer use through functional anchoring. With reference latents from information-balanced compression, CALR couples latent-mediated answer supervision with derivation-level semantic anchoring: the former routes answer supervision through intermediate states, while the latter grounds their decoded content in problem-specific derivations. A parallel-to-autoregressive curriculum develops sequential reasoning by conditioning subsequent latent blocks on generated prefixes. Evaluations on five mathematical reasoning benchmarks across model families show substantial accuracy gains. Under matched budgets, CALR gains 26.0 percentage points over a comparable continuous latent reasoning method. Further analyses show that its latents support answer prediction and carry problem-specific intermediate reasoning.