REER-PT: Reverse-Engineered Reasoning for Perplexity-Guided Pre-training Data Augmentation
2026-08-31 • Computation and Language
Computation and Language
AI summaryⓘ
The authors introduce REER-PT, a method to improve language model training by adding short explanations where the next words are hard to predict but can be logically inferred. These explanations help the model connect the context to what comes next without changing the usual training method. By applying REER-PT to training data, they observe that the model better predicts text and performs slightly better on tasks that require knowledge and reasoning. Their approach keeps the original text mostly intact and avoids extra computation during training.
language modelnext-token predictionpre-trainingperplexityreasoning annotationscontinuation predictiondata augmentationmodel evaluationreverse-engineered reasoningbenchmark tasks
Authors
Haoran Que, Jiajun Shi, Ting Huang, Renming Pang, Jiaheng Liu, Ge Zhang, Wenhao Huang, Shen Yan, Wei Ye, Shikun Zhang
Abstract
As language-model compute continues to scale, high-quality training data is becoming an increasingly important bottleneck. Conventional next-token prediction supervises what follows a context but leaves the intermediate reasoning behind that continuation implicit. We introduce \textbf{REER-PT}, a scalable framework that extends Reverse-Engineered Reasoning (REER) to raw pre-training data. REER-PT identifies continuations that are difficult to predict but can still be inferred from the preceding context, and inserts concise reasoning annotations that reconstruct the missing connection between context and continuation. Candidate annotations are generated and refined offline, with perplexity serving as the optimization signal. Constraints on length and target leakage filter out unhelpful or trivial annotations. This sparse transformation preserves the source text and remains compatible with standard next-token prediction, avoiding online reasoning rollouts during pre-training. We apply REER-PT to transform a source pre-training corpus into an augmented one. Across augmented-data, original-token, and selected-continuation comparisons, perplexity reductions range from 0.42 to 7.29, and only about 0.05\% of annotation 13-grams appear verbatim in the source text. We then train two 680M-parameter models with the same architecture and training configuration on the source and augmented corpora, respectively. The augmented-data model gains up to 2.07 percentage points on several knowledge and reasoning benchmarks. Together, the perplexity analysis indicates improved continuation predictability, while the controlled pre-training experiments suggest that this augmentation can improve model performance without changing the standard pre-training objective.