Random recursive models reuse layers flexibly to save parameters and improve reasoning
Random Recursive Models
Machine LearningArtificial Intelligence
Summary
Many AI models reuse the same parts repeatedly to keep smaller and simpler, but they usually do this in a fixed order. The authors introduce random recursive models (RRM) that pick layers at random each time they think, allowing them to reuse parts more flexibly while staying small. These models do well on tasks that need reasoning, often using much fewer parts. They can also think deeper than ever trained without changing, and use smart ways to guess answers that improve their performance.
What this means in practice
- •For machine learning engineers: Design smaller neural networks that match or exceed reasoning performance of larger models by using random layer reuse.
- •For ai system operators: Adjust model reasoning depth dynamically at inference without retraining to improve results on tasks needing deeper computation.
Authors
Jama Hussein Mohamud, Mirco Ravanelli
Abstract
Recursive models create computational depth through parameter reuse, offering a parameter-efficient alternative to increasing model size. However, most recursive models repeatedly apply one learned transformation or a prescribed sequence of transformations, restricting computation to a fixed layer order. We introduce the Random Recursive Model (RRM), which maintains a pool of $L$ learned layers and performs $T$ recursive steps by sampling one layer independently with replacement for each example and step. This enables flexible layer reuse while retaining the parameter efficiency of recurrence. We evaluate RRM on challenging reasoning tasks, where it matches or exceeds the baselines, often with 50-75 % fewer parameters. RRM can vary its depth at inference, including beyond that seen during training, without retraining or adding parameters, improving tasks that benefit from deeper iterative computation. RRM also supports Monte Carlo inference and probabilistic test-time scaling, both of which improve performance without retraining. These insights may open new directions in neural network architecture design.