Papers for

software engineers building ai assistants

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Language model predictions can match without matching belief changes

Same Answer, Different Future: Matched Predictions Do Not Imply Matched Belief Updating in Language Models

Abstract: Language models can produce similar probabilistic predictions after observing outcomes, reading summaries, receiving instructions, or undergoing activation interventions. Whether these routes induce equivalent belief states remains unclear: agreement in current predictions does not establish how a model will respond to subsequent evidence. We investigate this distinction in a Bernoulli inference task with an analytic Bayesian reference. Across three open-weight language models, we prepare states through four textual routes and two activation-level interventions, measure both next-outcome continuations and explicit probabilistic estimates, and append identical new evidence from each prepared state. We quantify the persistence of the initial belief through its effective evidence weight, the number of new observations it is worth. We find that the format of evidence decides which readouts express its content: outcome sequences primarily affect continuations, whereas summaries and stated rates primarily affect explicit estimates. Moreover, effective weights differ substantially across routes, with summaries receiving weights that scale weakly with their stated observation counts and elicitation questions changing inferred weights by orders of magnitude. In a one-shot steering setting, an intervention that reproduces a natural record's immediate prediction loses most of its effect under subsequent evidence. These results show that agreement in current readouts is insufficient to establish equivalence in belief updating: how a belief is acquired and how it is read jointly determine its subsequent dynamics.

Thu 5 NovMachine Learning
The gist
Language models often give similar answers when asked the same question in different ways, but this study shows that these similar answers don’t always mean the models have actually updated their beliefs the same way. The authors tested this using a simple probability task and found that how information is presented affects what the model thinks and says next. For example, showing examples versus summaries leads to different ways the model expresses its confidence and processes new information. So, just because two model states produce the same answer now doesn’t mean they will react the same to future information.
Open → 2611.07913v1

Context infrastructure reduces token use and latency in information retrieval loops

Context as Infrastructure: Knowledge Compilation for Retrieval in Agentic Systems with Harnesses

Abstract: Retrieval-augmented generation is becoming agentic. A search harness writes queries, reads results, reformulates, and decides when to stop. This largely solves accuracy: a harness with only BM25/grep and MiniMax M3 answers 85% of BrowseComp-Plus and 91% on multi-hop FRAMES in our runs, without compiled or vector indexes. It exposes a context problem: an undisciplined loop repeats queries, rereads documents, and fills the window with unused passages. The missing layer is knowledge compilation: a configurable ingestion pipeline where LLMs enrich documents with context no chunk contains alone--navigation hierarchies with claim-evidence leaves, entity graphs, timelines, mind maps, ontologies, wikis, and memory. Compilation is not a better retriever: under a competent harness, structural artifacts gave no detectable accuracy gain on FRAMES, and a PageIndex-style outline degraded at scale. It is a supply layer. A navigation tree whose leaves are claim-evidence pairs--the mechanism we measure--cuts query tokens by 91% when evidence suffices and adds 1.9% when it does not, a 38% average reduction; this implies a 2% break-even sufficiency rate and a measured rate near 43%. A decision gate cuts latency from 140 s to 30 s, though substitution's token saving is bought with retrieval rounds and carries its own latency cost. Other artifacts--ontology, graph, wiki, timeline, mind map, memory--pay off in deployment (governance, integration, onboarding, temporal orientation, continuity) where accuracy benchmarks cannot see them. We describe RAGFlow's pipeline and harness (open source at https://github.com/infiniflow/ragflow), which turns the loop into five disciplined stations and keeps the context entering the model the smallest sufficient set; together they form an agent-native database: a store that hands the agent compiled, governed context instead of a ranked list of passages.

Fri 30 OctInformation RetrievalArtificial IntelligenceComputation and Language
The gist
The paper studies how computers that gather and use information through repeated web searches can become faster and more efficient. The authors find that organizing and enriching information before searching—called knowledge compilation—does not directly improve accuracy but greatly reduces the amount of information the system must process and speeds up response time. They built a system called RAGFlow that breaks down the search and decision process into clear steps, giving the computer just the necessary information instead of many documents. This approach saves time and helps the system work more smoothly even when handling complex searches.
Open → 2610.41172v1

Large language models face challenges in understanding meaning rules

Rethinking Evaluation for Principled Generalization Using a Kripkean Challenge to Meaning as Use

Abstract: Large language models operationalize an intuitive, view of meaning as use. We argue that it faces a rule-following challenge. Using Kripke's interpretation of Wittgenstein, we argue that finite histories of use do not determine a unique rule for future extension. This reframes the debate about OOD generalization but also about NLP evaluation; benchmarks that measure performance on fixed use cases or local robustness, we argue, do not by themselves establish semantic competence. To evaluate whether models preserve rules and concepts under principled domain extension, we propose to evaluate two capacities, central to this ability: abstraction and recursion; and we offer an experimental paradigm for testing both.

Mon 26 OctComputation and Language
The gist
This paper discusses a problem with how large language models understand meaning based on how words are used. The authors argue that just looking at past examples can't tell a model the exact rules it should follow in new situations. They suggest current tests don’t fully show if models really understand language. Instead, the authors propose testing models on their ability to think abstractly and handle repeated rule applications to better measure real understanding.
Open → 2610.35111v1

KronLSTM improves long context memory with stable linear attention

Kronecker Delta Rule for Long Context Linear Attention

Abstract: We propose KronLSTM, a linear attention architecture based on the Kronecker Delta Rule, a novel update rule for associative matrix memories. KronLSTM extends the xLSTM family with a memory-revision mechanism that corrects targeted associations while modifying the memory only along a single direction in matrix space. This minimally invasive memory revision reduces interference with previously stored information compared with delta-rule updates, making it particularly well suited for long-context language modeling. The resulting memory dynamics can be mathematically described as a Markov chain that converges to a steady-state distribution, while the expected norm of the memory matrix remains uniformly bounded throughout the entire input sequence. Hence, KronLSTM ensures stable memory dynamics without requiring additional normalization and stabilization mechanisms for the memory state. Experiments show that KronLSTM is competitive with, and often outperforms, existing linear attention architectures, including models of the (Gated) DeltaNet family, with particularly strong performance on long context benchmarks. These results establish the Kronecker Delta Rule as an effective and analytically tractable foundation for stable associative memory in recurrent linear attention architectures. Our code (https://github.com/NX-AI/KronLSTM) and models (https://huggingface.co/collections/NX-AI/kronlstm) are open-source.

Mon 19 OctMachine LearningArtificial Intelligence
The gist
Long language sequences are hard for computers to remember without mixing up old information. The authors developed KronLSTM, a method that updates a computer's memory carefully so it doesn't forget or mess up previous memories. This approach keeps the memory steady, making it better for understanding long texts. Experiments show that KronLSTM often works better than other similar techniques, especially when dealing with very long information.
Open → 2610.26834v1

Improving multimodal reasoning accuracy with token contrast and confidence

Enhancing Multimodal RLVR with Pairwise Token Contrast and Adaptive Trajectory Confidence

Abstract: Reinforcement learning with verifiable rewards (RLVR) improves multimodal reasoning, but group-relative policy optimization broadcasts a single sequence-level advantage to every token of a response, although errors usually localize to a few decisive steps. Recent self-distilled variants refine credit with a teacher--student likelihood gap. However, this signal cannot tell whether it relies on the image or on a language shortcut, and is applied uniformly even when the teacher is uninformative. To address this, we propose Pairwise token contrast and Adaptive trajectory ConfidencE (PACE). Pairwise token contrast scores each response under four teacher views, crossing a correct or incorrect answer hint with the image present or removed, and compares the two hints for every token, which cancels shifts shared by both hints and reveals whether the token's support relies on the image. Adaptive trajectory confidence measures how well the teacher predicts each response given the correct hint and the image, normalizes this score within the batch, and uses it to set how far the token contrast redistributes credit within that response. PACE achieves the best average accuracy on three backbones and ranks first on all evaluated multimodal reasoning benchmarks with Qwen3-VL-8B-Instruct. Experiments show that both components matter: replacing pairwise token contrast with a single-view signal underperforms our method, while adaptive trajectory confidence adds further improvement.

Thu 15 OctComputation and Language
The gist
Reinforcement learning methods that guide multimodal models (those understanding images and text) often give credit to entire responses even when mistakes happen only in small parts. The authors propose a new technique called PACE, which looks at each word in a response from different perspectives—whether the answer is right or wrong, and whether the image is shown or not—to better understand how image information supports the answer. They also adjust how much the teacher model's confidence affects learning. This approach helps models learn more precisely where errors occur and improves their accuracy on multimodal reasoning tasks.
Open → 2610.21671v1

LLM planners balance safety and efficiency under uncertainty

Planning Commitment of LLM under Partial Observability

Abstract: LLM-based planning agents commit to plans of different lengths before interacting with a partially observable environment. Some aggressively commit to a long plan based on assumptions about unobserved states, while others conservatively plan only over what they have observed. Both behaviors emerge implicitly in pre-trained models, whether the LLM outputs a plan directly or generates a PDDL formulation for a classical solver. Aggressive commitment risks costly failures, while conservative commitment risks excessive LLM calls. This choice pits safety against efficiency, a trade-off that has been neither characterized systematically nor made controllable. We investigate aggressive and conservative LLM planners and formalizers on two partially observable domains, where priors are provided as either explicit statistics or common-sense knowledge. Across three LLMs, both strategies reach similar accuracy. Conservative methods achieve fewer failed attempts (safety) at the cost of more API calls (efficiency). Aggressive methods fail more often but replan efficiently only when priors are informative. Interestingly, analysis shows that LLMs treat the provided priors nearly as facts, which risks failure when priors are unreliable. We also explore hybrid strategies with tunable hyperparameters that adjust the balance between safety and efficiency.

Thu 15 OctArtificial IntelligenceComputation and Language
The gist
Planning with large language models (LLMs) in situations where not everything is known can be tricky. The authors show that some LLMs try to commit to a full plan quickly, which can lead to costly mistakes if assumptions are wrong, while others play it safe by only planning based on what they know, which uses more computing calls. Both strategies perform similarly overall, but each has clear trade-offs between making fewer errors and using fewer resources. They also find that LLMs often treat background knowledge as absolutely true, which can be risky if this knowledge is unreliable. The authors suggest adjustable strategies to balance safety and efficiency.
Open → 2610.21108v1

VisualCoding improves accuracy in visual reasoning tasks by learning reusable skill programs

VisualCoding: Learning Executable Skills for Reliable Visual Reasoning

Abstract: Learning reusable skills can improve visual reasoning agents, yet applying a skill still requires the agent to implement or invoke a procedure and produce the final answer. Errors and hallucinations at these steps can prevent acquired experience from translating into reliable single-run performance. We propose VisualCoding, which combines execution-guided acquisition of reusable visual programs with direct adoption of their answers on applicable inputs. Execution-Guided Program Acquisition composes visual primitives and additional code into family-level solvers, then iteratively tests and revises them on labeled training instances. Scope-Aware Direct Execution runs a retained solver and returns its valid in-scope answer. We evaluate VisualCoding on four models across four benchmarks spanning 69 task families. VisualCoding exceeds the strongest evaluated baseline in mean accuracy by 4.45 percentage points with Gemini-3.1-Pro and by 14.78 points when reusing the frozen Pro-acquired artifacts with Flash-Lite. Ablations support execution-guided acquisition and show that direct execution outperforms textual or callable reuse of the same programs. Code is available at https://github.com/jiaq-liu/VisualCoding.

Fri 9 OctComputer Vision and Pattern Recognition
The gist
Visual reasoning AI often struggles to give reliable answers because executing steps can lead to errors or made-up responses. The authors created VisualCoding, a method that teaches these AI systems to learn reusable visual skill programs and directly use their answers when applicable. This approach tests and refines skill programs during learning and then runs them carefully to avoid mistakes. Their method performed better than strong existing systems across many different visual reasoning challenges.
Open → 2610.14403v1

Large language model agents avoid tool misuse with new inference method

Understanding and Mitigating Hallucination Escape in Tool-Using LLM Agents

Abstract: Large language models (LLMs) increasingly serve as autonomous agents that invoke external tools. However, this capability introduces tool hallucination, selecting incorrect tools or generating invalid calls. Existing mitigation methods report substantial improvements, yet we identify a previously overlooked failure mode that we term Hallucination Escape. These methods reduce hallucination on the tool configuration they are tuned on but increase it on other configurations, canceling out the gain. We further investigate this phenomenon and find that hallucination rises sharply when a model's intrinsic tool-use tendencies conflict with the current tool configuration, and that existing methods reinforce rather than suppress these tendencies, which in turn contributes to hallucination escape. Building on these findings, we propose EscapeGuard, a training-free inference-time method that combines conflict-aware gating with configuration-derived attention enhancement to mitigate tool hallucination while preventing hallucination escape. Across six benchmarks on various models, EscapeGuard reduces tool-selection hallucination by 9.0 pp and suppresses hallucination escape, lowering the cross-configuration mean by 23.7 pp and achieving an 89.1% net improvement in paired-query evaluation. We hope this work can encourage evaluation beyond a single tool configuration and pave the way for more reliable tool-using LLM agents.

Sat 3 OctComputation and LanguageArtificial Intelligence
The gist
Large language models (LLMs) sometimes pick the wrong tools or make errors when using external tools because they imagine things that aren't true, known as hallucination. The authors found that previous fixes only reduce this problem for specific tool setups but make it worse for others—a failure they call Hallucination Escape. They discovered this happens when the model's natural preferences clash with the tool setup, and past fixes actually strengthen these biases. To fix this, they created EscapeGuard, a method that works during inference without retraining, helping LLMs choose tools correctly across different setups and greatly reducing hallucination errors.
Open → 2610.04409v1

Small language model agents learn better with harness-aware distillation

Harness-Aware Distillation for Small Language Model Agents

Abstract: Language model agents are deployed with a harness, the software around the model that manages its context, tools, and feedback. When such an agent is distilled into a smaller one, the harness stays in place, so the student mainly needs the teacher-specific abilities that the harness cannot provide, such as acting correctly on harness information. Standard distillation, however, imitates the teacher's full outputs and treats the harness as part of the input. We propose Harness-Aware Distillation (HAD), which focuses distillation on what the teacher adds beyond the harness. HAD complements on-policy distillation with two components: an action preference that contrasts the same teacher's actions with and without the harness information, scored after the student's own reasoning, and a validity check that drops preference pairs whose preferred action contradicts the harness records. We show that the contrast gives the student information that imitating the teacher alone cannot provide, and HAD needs no task rewards, success labels, or future information. Across multiple long-horizon agent benchmarks and models, HAD outperforms on-policy distillation baselines with the same fixed harness. Our analysis shows that HAD enters fewer unproductive loops and recovers from errors more often than the baselines, and suggests that it adaptively keeps learnable feedback in its weights while reading state information from the harness.

Fri 2 OctArtificial Intelligence
The gist
Large language models often work inside special software called a harness that helps manage their context and tools. When making smaller versions of these models, the usual way is to copy everything the big model does, including the harness’s effects. The authors of this paper found a better way to teach the small model by focusing only on what the big model adds beyond the harness. This method, called Harness-Aware Distillation, helps the small model learn to act correctly with the harness’s help without needing extra rewards or success signals. Their tests show this approach helps small models avoid mistakes and handle errors better.
Open → 2610.02858v1

Abstract pretraining improves reasoning beyond language fluency

Lasting Effects of Abstract Pretraining Beyond Perplexity

Abstract: Language models are typically pretrained from random initialization. Recent work challenges this convention, showing that a brief warm-up on abstract, algorithmically generated data can provide a better starting point for subsequent learning of natural language. In this paper, we show that in small language models, such a warm-up improves specific capabilities that are not reflected in language-modeling perplexity. Our warm-up uses an abstract stack-manipulation task that requires compositional and state-tracking capabilities. Allocating as little as 1% of pretraining tokens to this data improves multi-hop question answering by up to 3.9 F1 points on MUSIQUE, with additional gains on HOTPOTQA and 2WIKIMULTIHOPQA despite comparable language-modeling perplexity. Controlled experiments show that the warm-up substantially accelerates the acquisition of deeper reasoning chains. We also explore what drives this transfer. First, the structure of the data matters: replacing the stack task with a queue fails to produce the same gains. Second, the gains are specific: performance improves on sequential reasoning chains, with no consistent benefit on tasks that combine or compare independent facts. Third, timing matters: mixing abstract data with natural language is far less effective than an initial dedicated phase, and exposure after pretraining completely removes the benefits. The early advantage persists through billions of subsequent language tokens. These results show that early abstract training can reliably shape the capabilities language models later acquire.

Wed 30 SeptMachine Learning
The gist
Language models usually start learning from scratch with natural text. The authors found that giving models a brief early practice on a simple, rule-based task involving managing a stack of items helps them improve at complex multi-step reasoning questions later. This improvement happens even when the usual measure of language skill, perplexity, does not change. Their experiments show that the kind of early practice and when it happens matter for lasting benefits on reasoning tasks.
Open → 2609.38764v1

Exact method corrects bias in constrained text generation models

Twist, Don't Tilt: Trajectory-Exact Constrained Decoding for Masked Diffusion Models

Abstract: Constrained decoding for Masked Diffusion Language Models (MDLMs) aims to ensure that generated outputs satisfy a specified structure or syntax constraint. MDLMs generate outputs by repeatedly unmasking masked positions present in their current state. Recent strategies for constrained decoding constrain the model's per-step mean-field posterior (which factorizes over masked positions) by enforcing the desired constraint with an automaton. The resulting chain-structured factor graph allows exact constrained sampling via dynamic programming. However, despite each draw being exact and constraint-satisfying, we prove that their composition, in general, tilts away from the model's relative probabilities over valid trajectories, thus leading to trajectory bias. We derive an exact expression for this bias as a product of ratios measuring how valid continuation mass changes when the denoiser is reconditioned, and characterize when the bias vanishes. We then correct the bias by introducing TWISTER, the first automaton-twisted Sequential Monte Carlo decoder for MDLMs, using the step-exact decoder as the proposal. We show that for regular language constraints, the Feynman-Kac correction is exactly computable, with the twists obtained efficiently using quantities pre-computed for step-exact sampling. We prove that the resulting Feynman-Kac model targets the unbiased Doob h-transformed path law conditioned on constraint satisfaction.

Mon 28 SeptMachine LearningArtificial IntelligenceComputation and Language
The gist
Generating text with specific rules can be tricky for some AI models because the way they build sentences step-by-step can introduce subtle biases. The authors studied these biases in a type of masked diffusion language model, which fills in missing words repeatedly to create sentences that follow certain constraints. They found that previous methods caused a bias away from the model's true probabilities of valid word sequences. To fix this, the authors developed a new method called TWISTER that corrects the bias exactly, ensuring the generated text strictly follows the rules without skewing probabilities. This approach uses advanced sampling techniques to keep the outputs both valid and unbiased.
Open → 2609.35609v1

Large language models show big differences using similar tool interfaces

Action-Space Shaping for LLM Agents: Measuring and Mitigating Tool-Schema Bias

Abstract: Large Language Models (LLMs) have shown strong performance on tool-use agentic tasks when given a fixed tool schema. Yet a tool schema is not the action space of an agent; it is merely one interface representation of it. The same executable action can be exposed through many different, functionally equivalent tool definitions, and an agent that has truly learned a task should behave consistently across them. We show that current agents often do not, a phenomenon we term schema bias. To study this systematically, we introduce an executable transformation framework that rewrites a native tool schema using nine operators, including merging and splitting tools, altering how a single tool is expressed, and distributing one action across several dependent calls. The tasks, executable actions, and reachable states remain fixed, so any change in success is attributable to the interface alone. Evaluating eleven LLMs, including two closed models, on up to 32 schema variants, we ask how large schema bias is, how it manifests, whether the difficulty of a schema variant can be predicted without a full evaluation, and whether training removes it. We find that schema bias is substantial even for the newest models: success rates range from complete failure to 97% depending solely on the schema. To reliably estimate schema difficulty, it requires running a small sample of the target queries. Training repairs a schema variant only when that variant appears in the training data.

Mon 28 SeptArtificial Intelligence
The gist
When large language models (LLMs) use tools through different but equivalent interfaces, they often perform inconsistently. This problem, called schema bias, means that changing how a tool’s actions are presented can cause the same model to fail or succeed dramatically. The authors tested many LLMs with variations of tool interfaces that do the same thing but look different, finding large performance swings. They also showed that training a model helps only if the exact altered interface is in the training data. This reveals that LLMs do not fully understand the underlying tasks but rely heavily on how tool actions are shown.
Open → 2609.34971v1

Tessera cuts delays in memory-heavy AI model requests by up to 3.6 times

Tessera: Demand-Driven KV Cache Management for Retrieval-Augmented LLM Serving

Abstract: RAG and retrieval-based agent memory both inject retrieved content into LLM prompts, as document chunks and recalled memory records, respectively. The same content can recur across requests at different prompt positions or after different preceding contexts, preventing reuse through conventional prefix caching. Our characterization finds that records recurring outside the matching prefix account for over 70% of injected memory tokens in agent-memory workloads. Composable KV-reuse methods enable reuse in such cases, but online serving introduces a management problem: a recurring unit's KV states may not yet exist, may have been evicted, or may reside on another node. We present Tessera, a disaggregated serving system that makes retrieval the control plane for KV reuse. By exposing the context units needed before model execution, retrieval allows Tessera to combine current demand with retrieval history, KV residency, and generation load to coordinate cache management and request routing. Generation nodes concurrently prepare locally cached, remotely cached, and missing states, while retaining newly computed states off the request's critical path. Across RAG and agent-memory workloads, Tessera lowers mean TTFT by up to 3.6x over SGLang and LMCache with EPIC at matched request rates, and sustains low TTFT at rates where the baselines saturate, while matching the answer quality of the underlying composition policy.

Sat 26 SeptDistributed, Parallel, and Cluster Computing
The gist
Large language models (LLMs) often use extra information from documents or memory to answer questions better. However, reusing parts of previous computations to save time is tricky because this useful content can appear in different places or orders, making simple caching ineffective. The authors designed Tessera, a system that smartly manages caching by knowing what pieces of information are really needed upfront and sharing that knowledge across servers to prepare answers faster. Tessera speeds up response times significantly while keeping answer quality high, especially when many users ask questions at once.
Open → 2609.32999v1

Logical reasoning resides in a small brainlike part of language models

Logical subspace in LLMs

Abstract: Recent work has identified a human brain network specialized for abstract formal reasoning (Kean et al., 2025). Does the same hold true in language models? To answer this question, we introduce the minimal viable subspace (MVS) method, which searches for the lowest-rank activation subspace at a layer that preserves task performance when everything outside that subspace is ablated. Using MVS, we demonstrate low-rank subspaces supporting logical inference on Gemma and Qwen models. Furthermore, these subspaces exhibit a clear dissociation from model capacities on other tasks, such that retaining these late logic subspaces preserves inference while impairing factual knowledge, working memory, cognitive control, and arithmetic. Conversely, ablating them reduces logical inference accuracy to chance while largely sparing these other capacities. Our results suggest a functionally localizable core machinery for logic akin to that in the human brain.

Sat 26 SeptArtificial IntelligenceComputation and Language
The gist
Finding out whether language models have special parts for logical thinking like the human brain, the authors devised a way to pinpoint tiny areas in the model’s ‘brain’ that do the thinking. They found that these small parts handle logic tasks without helping with other abilities like memory or math. Removing these parts breaks logical thinking but leaves other skills intact. This shows that logical reasoning is localized in language models just as in human brains.
Open → 2609.32907v1

Language models lose structure when communicating complex expressions

The Communication Bottleneck: A Round-Trip Study of Tree-Structured Expression Serialization in Language Models

Abstract: When language models reason in chain-of-thought or exchange free-text intermediates, they serialize structured information into natural language. How much tree-structured compositional content survives this bottleneck? We propose a round-trip protocol that answers this question empirically for tree-structured expressions. A generator converts a procedurally generated arithmetic expression into a word problem, a separate extractor recovers the expression from the word problem alone, and symbolic equivalence provides an exact oracle. Evaluating all pairwise combinations of sixteen models yields a communication matrix whose marginals separate generation quality from extraction quality. Three main findings emerge. First, the channel is lossy and asymmetric: swapping which model generates and which extracts shifts accuracy by up to 60.4 points, and the best pair reaches 92.9% by combining different models on each end rather than the same model on both. Second, at least 73.6% of round-trip failures originate at generation, and difficulty is driven by tree structure (operator count, depth, right-branching) rather than model family. Third, the channel is trainable: ~3600 fine-tuning examples that share the evaluation's operators and tree shapes lift every open-weight model above untrained Gemini-3.1-Pro, an upper bound under matched semantics. A disjoint-domain regime with new operators and vocabulary also raises every open-weight model, confirming the gain is not an artifact of matched semantics, though a gap to the frontier remains. Together these results identify tree-structured expression serialization as a primary limiting factor when models communicate hierarchical structure through natural language.

Fri 18 SeptArtificial Intelligence
The gist
Language models often translate complicated tree-like math expressions into text and then try to recover the original. This process is imperfect and means some details are lost or changed. The authors show that different models vary a lot in how well they can generate and understand these texts, and that training on similar examples helps. This loss limits how well models can share complex structured ideas through plain language.
Open → 2609.21509v1

Demonstration selection made efficient using state space models

Long-Context Demonstration Selection Using State Space Models

Abstract: We study the problem of demonstration selection, which involves selecting a subset of examples for prepending to a query to a language model. This problem is closely related to in-context learning and language model inference. Since the inference cost of a transformer model scales quadratically with sequence length, the selection problem becomes especially challenging in a long-context scenario. In this paper, we tackle this problem by building on state space models (SSMs), which require only linear inference time given the input. Our approach involves two algorithms. The first learns a small set of SSMs through distillation of a (trained) transformer model. We partition all the layers into consecutive groups. Then for each group, we estimate a separate state space model to replicate the input-output behavior within the adjacent layers. Second, we map the distilled model outputs to a small set of tokens, and apply these embeddings for demonstration selection in downstream applications. We perform extensive experiments in both synthetic and real-world datasets to validate our approach. We demonstrate that the distilled SSMs only incur an approximation error of less than $0.7\%$ relative to the true output. In downstream evaluation, we show that on several text classification and reasoning tasks, our approach reduces FLOPs by $14.2\times$ and improves accuracy by $6.48\%$ relative to baseline demonstration selection methods.

Tue 15 SeptMachine LearningComputation and Language
The gist
Choosing examples to show a language model before asking it a question can be slow and costly when the examples are long. The authors developed a method that uses simpler mathematical models called state space models to imitate parts of the original complex model more quickly. This lets them pick helpful examples faster and use less computing power without losing accuracy. Their tests showed this method works well on text classification and reasoning tasks, making the process both faster and more accurate.
Open → 2609.17888v1