Summary
Large language models (LLMs) usually run through all their layers the same way every time, no matter how easy or hard a question is. Inspired by how the human brain routes information differently based on what’s needed, the authors study a method called PoLar that lets the model skip or repeat parts of its layers for each input. They confirmed that skipping and repeating layers can improve accuracy, and that easier questions need fewer layers than harder ones. However, the part that chooses the best path in one try often defaults to the normal process, and the programs that fix errors are fragile. They also show that just a few common layer programs solve most questions and compare this process to how the brain’s cortex works.
What this means in practice
- •For machine learning developers: Design transformer-based systems that dynamically skip or repeat layers to improve question-answering accuracy depending on input difficulty.
- •For ai infrastructure engineers: Implement adaptive routing to reduce unnecessary computation on easier model inputs, optimizing resource use without accuracy loss.
Authors
Justus Westerhoff, Stephan Olbrich, Hatem Oraby, Matthew Evan Larkum, Felix Alexander Gers
Abstract
Inference in LLMs is conventionally a fixed-depth, fixed-order forward pass through every layer, regardless of how difficult the input is. The human brain does not work this way: using the thalamus as a central hub, it routes information flexibly to all regions of the cortex according to demand. Li et al. (2026) recently showed, with a system they call program-of-layers (PoLar), that transformers can be given an analogous flexibility if their layers are treated as a library of functions rather than a fixed sequence. Performance improves over the standard forward pass when each input is dynamically routed through an adaptive sequence of skipped or repeated contiguous layer blocks. We reconstructed PoLar's diagnostic MCTS in more detail than the original paper and applied it across 5 models. We reproduced several of PoLar's findings: skipping outperformed the standard pass, repeating outperformed skipping, and combining both outperformed either alone. Shorter programs sufficed for easier questions, while harder questions required more layer repeats. However, we failed to replicate the main claim regarding their learned router for single-shot inference: its top-ranked prediction consistently collapsed back to the standard pass, even though its top-k predicted programs, taken together, did show a real accuracy gain. Beyond reproduction, we find that a small number of generic programs are enough to solve most of the questions. We also provide a much deeper analysis of these programs' structure and robustness: for example, we found that programs that correct errors are highly brittle: undoing even a single edit inside a program typically breaks the correction. Connecting this to the brain's routing mechanisms, PoLar mirrors principles of thalamo-cortical coordination between cortical-area-like transformer layers. We publicly release the code at https://datexis.github.io/RE-PoLar/