Papers for

language ai engineers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Adaptive tests speed up language model answer selection with error guarantees

Sequential Anytime Valid Tests for Efficient Inference-Time Alignment

Abstract: Inference-time alignment methods, such as Best-of-N (BoN), improve language model outputs by generating multiple candidates per prompt and selecting among them using a reward model. Much of this cost is wasted: a fixed budget is applied to every prompt, although the computation required by prompts varies widely. Adaptive methods halt once sampled answers appear settled, but their thresholds are tuned heuristics with no statistical control over early-stopping errors. We instead seek an anytime-valid stopping rule that takes an error tolerance $α$ as input and halts only once an answer is certified to hold a majority ($θ_a > 1/2$) under the generator's distribution. We aggregate candidates by final answer with a reward-weighted self-consistency (RWSC) score and treat stopping as a sequential test over the resulting clusters. Applying Wald's sequential probability ratio test to these clusters yields SPRT-RWSC, which performs well empirically, but its guarantee fails because the leading answer shifts during sampling, and the run is continuously monitored. Our main contribution, e-RWSC, replaces Wald's test with a betting supermartingale (an e-process) for every answer the policy produces, bounding the probability of ever certifying a non-dominant answer by $α$, uniformly over time and across all answers. The RWSC scores drive the bets, so they affect only how quickly a dominant answer $a^\star$ is certified, not validity; certification takes $O(\log(1/α)/d_{\mathrm{KL}}(θ_{a^\star}\|1/2))$ samples. Experiments across several language models, datasets, and reward models confirm the predicted $1/d_{\mathrm{KL}}$ scaling of cost and show that accuracy improves over baselines at substantially lower token cost.

Thu 19 NovMachine Learning
The gist
Choosing the best answer from multiple options generated by a language model can be slow because traditional methods spend the same amount of time on every question. The authors developed a new testing method that decides when to stop looking for better answers based on a formal error tolerance, ensuring the most likely correct answer is selected without wasting effort. Their method uses a sophisticated mathematical approach called a betting supermartingale to provide strong guarantees on the correctness of the chosen answer while saving computational resources. Experiments show this new approach is more efficient and accurate than previous stopping strategies.
Open → 2611.26413v1

Language models use small word sets in poetry despite large vocabularies

Language Models are Lazy Poets: Creative Mode Collapse at Scale

Abstract: While aligned large language models (LLMs) can successfully complete many creative tasks, their outputs tend to be highly self-similar. Though solutions to this problem of "mode collapse" have been proposed, comprehensive large-scale surveys of the phenomenon in specific textual genres are still lacking. We address this gap through an extensive treatment of LLM mode collapse in poetry. Across 15 poetic forms, 248 subject matter tags, and 16 open-weight models, we generate over 1.1 million poems. To our knowledge, this constitutes the largest corpus of LLM-generated poetry to date. At million-poem scale, we find that LLM poetic vocabularies and rhyming abilities are substantial, but severely underutilized. In other words, LLMs are "lazy" poets: despite possessing large poetic inventories, LLMs over-rely on a small number of words, opening phrases, and rhymes. Further, we find that lexical homogenization differs across models and poetic forms, noting observed rhyming vocabulary is correlated with model size.

Fri 16 OctComputation and Language
The gist
Large language models can create poems but tend to repeat the same words, phrases, and rhymes over and over, even though they know many more. The authors studied over a million poems generated in many styles and found that this 'laziness' varies by model and type of poem. Bigger models tend to use a larger rhyme vocabulary but still rely on a limited subset of language. This means that current models are less creative in poetry than one might expect given their capabilities.
Open → 2610.24223v1