Papers for

autonomous robotics teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Gaussian processes improve path planning accuracy and efficiency

Informative Path Planning with Gaussian Processes: Error Bounds, Algorithms, and Benchmarks

Abstract: Accurate environmental monitoring relies on Informative Path Planning (IPP) to deploy autonomous systems that maximize information gain. While Gaussian Processes (GPs) provide a robust mathematical foundation for modeling the environment, contemporary IPP approaches remain hindered by heuristic operational constraints and the lack of simultaneous theoretical guarantees for worst-case estimation error and approximation quality. Rather than introducing yet another heuristic, this paper presents a rigorous theoretical analysis of existing error bounds and algorithms to further develop the mathematical foundation for IPP. By evaluating probabilistic and deterministic bounds on reconstruction error, we establish that predictive variance is the theoretically ideal optimization objective for GP-based IPP. Furthermore, we analyze GreedyCover, a recent approach for optimizing predictive variance, and demonstrate that its near-optimal approximation guarantee matches the fundamental hardness-of-approximation limit for the problem. Comprehensive benchmarking confirms that this approach delivers superior estimation accuracy and computational efficiency compared to prominent information-theoretic objectives and optimization methods used in IPP. Finally, this theoretical analysis establishes a rigorous, near-optimal foundation for discrete offline IPP under bounded uncertainty constraints, offering clear and principled guidance for future work in continuous-space and online formulations.

Thu 12 NovRobotics
The gist
Collecting accurate environmental data is challenging, especially when using robots that move around to gather information. The authors analyze how to plan paths that maximize useful data gathered by focusing on a mathematical model called Gaussian Processes. They show that optimizing uncertainty predictions leads to better data collection than previous methods. Their work also proves that a recently developed planning algorithm is nearly the best possible in terms of accuracy and speed. This provides clear guidance for future improvements in robotic environmental monitoring.
Open → 2611.16877v1

Functional task networks improve learning without forgetting in neural nets

Autonomous Continual Learning and Knowledge Transfer via Cortex-Inspired Neural Assemblies

Abstract: Unlike the brain, artificial neural networks (ANNs) suffer from "catastrophic forgetting": complete overwriting of solutions when re-training on new functions ("concept shift") or on a disjoint domain of the current function ("domain shift"). ANNs require each batch of data to be roughly representative of the complete distribution, a typically expensive and impractical requirement. In this paper, we introduce Functional Task Networks (FTNs), a method of continual learning and knowledge transfer modeled after several properties of the mammalian neocortex. FTN is formulated as a distributed, stochastic mixture of experts in which a sparse, binary routing mask is computed at inference time using a small support sample of input-output pairs. Per-batch reconfiguration of the router during training, together with mask-overlap-filtered replay, shapes a basis of overlapping, partially trained subnetworks. Unrelated tasks route through mostly separate assemblies, related tasks overlap on shared components. On block-sequentially trained continual learning and knowledge transfer tasks FTN retains solutions nearly as well as disjoint-mask isolation and attains the lowest pre-update head-start loss among matched-architecture comparators. The learned mask-overlap geometry tracks the known concept-sharing structure where available. We show experimentally that under correlated concept shift, support-set mask reconfiguration is necessary even for retention. Under domain shift, the distributed bank with replay is sufficient to handle retention without the configurer. Forward transfer, or head-start, and backward transfer, are also contributed by support-set mask configuration.

Mon 9 NovMachine Learning
The gist
Artificial neural networks often forget old skills when learning new ones, unlike the human brain. The authors created Functional Task Networks (FTNs), inspired by the brain's neocortex, to help networks remember previous tasks while learning new ones. FTNs organize small subnetworks that can overlap for related tasks and separate for unrelated ones, improving memory and knowledge transfer. This method helps networks adapt better to changing tasks without needing large balanced datasets all the time.
Open → 2611.11717v1

Map-anchored prompting doubles drone success in object finding

Reaching Is Not Finding: Map-Anchored Visual Prompting for the Last Mile of Aerial Vision-Language Navigation

Abstract: Aerial vision-language navigation (VLN) asks a drone to find an object described in natural language. The description usually locates the object relative to named landmarks, such as streets and buildings. We observe that current navigators often bring the drone close to the target and then stop at the wrong place: they reach the target without finding it. This gap is shared by navigators of very different designs, from imitation learners to a fine-tuned 7B vision-language model. A simple test reveals the cause. When we replace the target object in the instruction with another one next to the same landmarks, no navigator goes to it, and most stop where they did before. Navigators learn to reach the landmarks, not to identify the described object. We therefore propose MAP-VP, a plug-and-play module for the last step. After any navigator stops, MAP-VP draws the landmarks named in the instruction onto the drone's current view, using the landmark outlines that the benchmark provides on a map. A fine-tuned 4B vision-language model then locates the target relative to these landmarks. The training data are generated automatically from the benchmark annotations. Neither drawing the landmarks nor fine-tuning helps much alone, but together they are highly effective. Without retraining any navigator, MAP-VP significantly improves all six navigators we test on CityNav. It roughly doubles the success rate of the two strongest trained navigators, raising HETT from 25.9% to 54.6% and FlightGPT from 24.2% to 48.9%, whereas CityNav's official detector-based last step adds less than two points to HETT.

Sun 8 NovComputer Vision and Pattern Recognition
The gist
Drones often get close to a target but fail to find the exact object described in natural language, stopping near landmarks instead. The authors found this happens because current systems learn to reach landmarks rather than identify specific objects. They created a module called MAP-VP that draws landmarks on the drone's visual feed and uses a smaller vision-language model to pinpoint the exact target relative to these landmarks. This method significantly improves the drone's ability to find objects without retraining the original navigation systems.
Open → 2611.11510v1

Evaluating hierarchical decision policies from flat data cuts bias

Intra-Option Fitted Q-Evaluation: Evaluating Hierarchical Policies from Non-Hierarchical Data

Abstract: Off-policy evaluation (OPE) estimates the expected return of a target policy from previously collected data without additional environment interaction. While OPE methods for flat Markovian policies are well studied, little work has addressed evaluating hierarchical policies in which a high-level policy selects temporally extended options that generate primitive actions until a termination condition is met. The primary existing approach applies per-decision importance sampling at the option level, but requires option-annotated trajectories and exhibits variance that still grows with the horizon. For non-hierarchical policies, fitted Q-evaluation (FQE) typically achieves lower error than importance sampling by leveraging the classic Bellman equation for policy evaluation; however, a naive extension of FQE to hierarchical policies results in a biased policy value estimate. In this paper, we introduce Intra-Option Fitted Q-Evaluation (IO-FQE), which estimates a value function in the augmented state-option space; this approach enables OPE of hierarchical policies even when we lack annotations of what option was run in the data. We develop two continuous-action instantiations of IO-FQE and show on long-horizon hierarchical continuous control tasks (AntMaze navigation and OGBench Puzzle manipulation) that IO-FQE eliminates the bias incurred by a naive application of FQE to hierarchical policies and substantially lowers mean squared-error compared to option-level importance sampling.

Sat 7 NovMachine Learning
The gist
Estimating how well complex decision-making plans work without testing them in real life is hard, especially when these plans include several steps that last different amounts of time. The authors found that existing methods that try to evaluate such multi-step plans often need extra information or make errors that grow the longer the plan is. They developed a new technique called Intra-Option Fitted Q-Evaluation (IO-FQE) that can accurately assess these plans using ordinary data that doesn't label each step. Their method reduces errors and bias, demonstrating better performance on tasks like navigating mazes or solving puzzles using robots.
Open → 2611.10543v1

Predictive models struggle to learn cause effects from data alone

Predictive gains, causal losses: why interventional competence cannot be learned from data alone

Abstract: Retaining predictive gains can require persistent errors in a model's response to interventions. For deterministic predictors with finite second moments under squared-error loss, any observational advantage over the best predictor using only an outcome's direct causes requires dependence on other inputs, regardless of the learning rule. When admissible interventions vary inputs while preserving the worldly outcome, this dependence yields a positive lower bound on root-mean-square intervention-response error. We identify conditions under which population gradient updates introduce this dependence from a parent-only baseline or damage a previously correct mechanism through shared parameters while improving other predictions. These dependencies can also make model behavior vary with how the same intervention is realized. We characterize interventional alignment through a common output mapping across internal states and physical processes realizing the same intervention target and relevant context. Correction is limited by the causal hypotheses currently admitted within inquiry. The relevant hypothesis class comprises these causal alternatives, rather than the function class the computational system could in principle represent. For predictions restricted to these alternatives, interventional mean error cannot fall below the best approximation they permit, whether evidence is observational or interventional. Reducing error below a positive limit requires admitting better hypotheses, which may entail revising the causal variables, mechanism structure, or intervention interpretation. Interventional criticism constrains replacements without determining them. We conjecture that reliably acquiring and maintaining interventional competence when causal commitments fail requires autonomous participation in proposing alternatives, testing their consequences, and revising them in response to error.

Mon 2 NovArtificial IntelligenceMachine Learning
The gist
Predicting what will happen when you change something in the world is different from just guessing based on past observations. The authors show that models can be good at prediction but still make mistakes when trying to understand cause and effect if they only learn from data without knowing the right causal story. They explain why it's impossible for such models to always get cause-and-effect relationships right unless they consider better explanations about how the world works. The authors suggest that to truly understand interventions, models need to actively test and update their ideas about how things cause one another.
Open → 2611.03861v1

Algorithm achieves optimal learning speed in episodic reinforcement learning

Closing the Log Gap: Minimax-Optimal Regret for Episodic RL

Abstract: In this paper, we study the regret minimization problem in episodic tabular MDPs. Our algorithm achieves the first expected regret bound scaling as $\sqrt{H^3 S A K}$ for sufficiently large $K$, where $H$ is the episode length, $K$ is the number of episodes, and $S$ and $A$ are the numbers of states and actions, respectively. Our method is the first to match the minimax lower bound \emph{without extra polylog factors}. We introduce a variant of UCBVI which does not maintain strictly optimistic estimates, and uses MOSS-style exploration bonuses whose logarithmic terms adapt to both visit counts and estimated variances. A key challenge is controlling the propagation of errors caused by under-optimism, which we accomplish in an occupancy-weighted way, avoiding logarithmic overhead that would be incurred by uniform-over-rows concentration. Our analysis develops time-uniform and variance-aware concentration bounds that may be of independent interest. As a corollary we also develop an algorithm for $(\varepsilon, δ)$-PAC best-policy identification, which is the first to match the $Ω\big(H^3 SA\log(\frac{1}δ)\varepsilon^{-2}\big)$ minimax lower bound up to constants in some regimes.

Thu 29 OctMachine Learning
The gist
This paper addresses how to make computers learn the best actions in a limited number of tries within a game or task that has many steps. The authors developed an algorithm that learns almost as fast as theoretically possible, without extra slowdowns usually caused by complicated calculations. Their approach adjusts how it explores the task, balancing risk and uncertainty to improve learning speed. They also provided a new method to identify the best strategy reliably with fewer samples than previous methods.
Open → 2610.40165v1

Vision guided robot navigation learns to plan and act together

MATCH: Execution-Aware Hierarchical Reinforcement Learning for Vision-and-Language Navigation in Continuous Environment

Abstract: Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires instruction-consistent planning and reliable execution under collision constraints. Yet many hierarchical approaches optimize the planner while keeping execution fixed. We introduce MATCH, a framework that jointly learns a language-conditioned planner and a goal-conditioned primitive-action executor. An execution-aware topological graph represents start-dependent execution options and integrates executor value estimates with realized outcome memory, enabling the planner to select execution requests and refine their targets through bounded continuous offsets. A transition-aware dense macro reward balances ordered progress along the reference route against execution residual, encouraging executable requests without substituting local success for instruction following. A staged curriculum culminates in synchronized on-policy updates, with the executor learning on planner-generated requests and the planner adapting its requests to the executor's current behavior. With 0.5B parameters, MATCH achieves state-of-the-art performance on the R2R-CE and RxR-CE benchmarks. Preliminary real-robot deployments further demonstrate its practical potential.

Wed 28 OctRobotics
The gist
Robots that navigate by following spoken or written instructions need to both plan routes and avoid bumps or obstacles while moving. This paper presents MATCH, a method that trains a robot’s planning and movement parts together, so the robot can better understand instructions and move more safely and efficiently. The authors show that MATCH works well in simulated tests and even in experiments with real robots, proving its promise for practical use.
Open → 2610.38248v1

Reflection-guided learning improves long-term decision making in AI agents

ReflectPO: Reflection-Guided Policy Optimization for Long-Horizon LLM Agents

Abstract: With external tools, large language models (LLMs) can tackle complex tasks through sustained environmental interaction. LLM-based agents are therefore increasingly used to tackle complex long-horizon tasks in real-world environments. However, these tasks involve sequential, interdependent decisions and changes in environmental states. Early execution errors affect subsequent interaction histories and environmental states, and their effects accumulate along the decision chain. To address this challenge, we propose Reflection-Guided Policy Optimization (ReflectPO), a reinforcement learning framework for autonomous error recovery in long-horizon tasks. ReflectPO comprises two core components. The first is Reflection-Guided Sampling. We use branch differences and repeated visits in interaction trajectories to construct hints for error recovery. The second is Tree-Structured Credit Assignment. We organize sampled trajectories into trees and assign action-level credit using parent--child reward differences and recovery gains. Experiments over two long-horizon tool-use benchmarks show that ReflectPO outperforms multiple existing baselines in most domains. It also achieves the best overall performance on average. Further analyses show that ReflectPO effectively improves agents' ability to recognize their own deviations, recover from erroneous states, and continue making progress toward task completion.

Thu 22 OctComputation and Language
The gist
Making good decisions over a long time is hard for AI agents because early mistakes can mess up everything that comes after. The researchers created a method called ReflectPO to help AI better recognize when they have gone off track and fix their mistakes along the way. ReflectPO uses smart ways to look back at previous actions and rewards to figure out which decisions led to errors and how to recover from them. Their tests show that agents using ReflectPO do better at completing long tasks with tools than other existing methods.
Open → 2610.30933v1

LLM agents struggle to adapt and explore in changing environments

AdaExploreBench: Benchmarking LLM Agents for Proactive Exploration and Test-time Adaptation in Evolving Environments

Abstract: Large Language Model (LLM) agents are increasingly deployed in unfamiliar and evolving environments, requiring them to proactively explore, understand, and adapt to environmental dynamics. However, existing benchmarks rarely test whether agents can proactively discover how an environment works and adapt as it evolves. To address this gap, we introduce AdaExploreBench, a benchmark for proactive exploration and test-time adaptation in evolving environments. By instantiating environmental dynamics as hidden rules, AdaExploreBench requires agents to actively acquire evidence and infer the rules. Under implicit rule drift, the latent rule changes without announcement, requiring agents to detect from degraded performance that prior knowledge is stale and re-explore. Experiments on eight leading LLMs show that proactive and adaptive exploration remain challenging: GPT-6 Sol achieves only 61.9% calibrated accuracy, while providing sufficient evidence or removing rule drift improves average performance significantly by close to 10%. These results reveal a critical bottleneck in current LLM agents: they can often reason effectively from informative evidence once it is available, but struggle to actively acquire the right evidence and recognize when previously acquired evidence is no longer valid.

Mon 19 OctComputation and LanguageArtificial Intelligence
The gist
Large language model agents often have to work in places they don’t fully understand and that can change over time. The authors created a new test called AdaExploreBench to see how well these agents can figure out hidden rules in these environments and adapt when the rules change without warning. They found that even the best models still have a hard time exploring effectively and realizing when their previous knowledge is outdated. This means current AI agents can use information well once they have it but struggle to find the right information or notice when it becomes wrong.
Open → 2610.25950v1

Reinforcement learning algorithms improve themselves with ai guidance

RLDISCOVER: LLM-driven co-evolution of reinforcement learning algorithms

Abstract: LLM-guided program evolution has enabled discoveries in mathematics and computational optimization, raising the prospect of reinforcement learning (RL) algorithms that self-evolve to improve how agents learn. However, realizing this prospect faces two obstacles. Joint search over coupled algorithmic components is difficult to scale: simultaneous changes can disrupt learning, while isolated changes overlook their dependencies. Evaluating candidate algorithms also requires costly training, with fitness remaining uncertain across random seeds. We introduce RLDiscover, a framework for the self-evolution of model-free deep RL algorithms. Progressive Co-Evolution advances from targeted component edits to joint evolution, while Progressive Probabilistic Evaluation balances search breadth and evaluation fidelity through staged training and repeated evaluation. Experiments across SAC, PPO, and DQN on four benchmark suites show substantial improvements in mean return, with per-family median gains of 32%-84% and a peak return ratio of approximately 363x over a near-zero baseline. These gains include transitions from failed learning to successful task completion, and improvements persist when evolution starts from stronger open-source implementations. On measured SAC locomotion runs, evaluation uses approximately one-fifteenth the estimated compute required to fully evaluate the same candidate pool. Remarkably, independent searches repeatedly discover interpretable combinations of adaptive robust losses, progress-dependent value targets, and running statistics, with selected programs transferring to unseen tasks. These findings point toward a broader role for self-evolution in AI: discovering interpretable algorithms that improve how agents learn.

Tue 6 OctArtificial Intelligence
The gist
Finding better ways for AI agents to learn is hard because changing many parts at once can confuse the system. The authors created RLDiscover, a method where AI slowly tweaks its own learning steps and checks if those changes help by trying them out carefully. Their tests show big improvements in popular AI learning methods, using much less computing power than usual. The new methods even work well on new tasks, highlighting potential for AI to improve its own learning processes over time.
Open → 2610.09218v1

HARP improves coordination of AI agents with hidden preferences

Large Language Model Orchestration under Heterogeneous Preferences via Explicit Persona Inference

Abstract: LLM orchestration investigates how an orchestrator coordinates a group of autonomous agents to achieve common goals or maximize collective welfare. The agents are typically heterogeneous, each holding a private preference that it pursues but does not reveal. Inferring such hidden preferences from behavior has been a subject of long-standing research in game theory and multi-agent systems. The core challenge lies in maintaining a belief over every agent's preference and updating it from the agents' observed actions. Existing LLM orchestrators carry that belief as prompt text with no explicit update rule. This lets early errors persist and propagate rather than be corrected. We therefore propose \textbf{HARP} (Heterogeneous-preference Agent oRchestration via Preference inference), a novel framework that moves the belief out of the prompt. Specifically, HARP maintains one numeric posterior per agent over a finite set of candidate preferences and updates it in closed form by Bayes' rule. The language model supplies only actions and per-candidate likelihoods, so estimation is decoupled from its reasoning. We prove that HARP attains the same $\tilde O(\sqrt K)$ Bayesian regret as explicit joint inference when the factorization is exact. Furthermore, HARP\textsuperscript{+} augments planning with a bonus for actions that distinguish the candidates, so inference continues even when the optimal action is uninformative. Empirical results on three substrates, ranging from payoffs the preferences fully determine, through payoffs that depend on more than them, to scales where explicit joint inference is infeasible, demonstrate that HARP\textsuperscript{+} is the strongest non-oracle method across the class our theory identifies.

Tue 6 OctComputation and Language
The gist
Sometimes AI agents have private likes or goals that they don’t say out loud, making it hard to predict what they will do. The authors develop a new method called HARP, which guesses each agent’s preferences more clearly by keeping numbers about likely preferences and updating them logically. This helps the AI team work better together by fixing early mistakes quickly. They also enhance HARP to keep learning even when actions don’t clearly show preferences. Tests show their method works well across different situations and scales.
Open → 2610.07587v1

Skill-following agents execute complex tasks through step-by-step contracts

Beyond Instruction Following: Learning Grounded Skill-Following with Skill Contracts

Abstract: Instruction following typically enforces discrete, response-level requirements, whereas an expert-authored skill prescribes procedural requirements spanning multiple phases and environment interactions. Given such a skill, we train the executor to execute all required phases instead of focusing solely on the final answer. We therefore introduce Grounded Skill-Following, which requires an agent to execute a fixed, expert-authored skill across its required phases by grounding decisions in environment observations. To achieve verifiable procedural execution, we formulate each skill as a skill contract combining visible skill instructions with an explicit contract runtime. The runtime specifies required phases, admissible actions, permitted transitions, and accepted termination. This structure provides a dense, verifiable training signal throughout execution. We leverage this by introducing Verified Progress Credit, which assigns rewards upon the initial completion of contract milestones and aggregates them into the trajectory return to guide policy optimization. During rollout, the contract runtime continuously tracks state transitions to provide Contract-State Feedback, which indicates whether the latest action is accepted and guides the agent toward valid next actions. To measure procedural compliance, we introduce the Protocol Completion Rate (PCR), defined as reaching accepted termination through all required phases, and decouple it from the final Task Outcome. Jointly trained with our framework, Qwen3.5-4B achieves Protocol Completion Rates of 99.27% on Math and 99.96% on Search, while slightly outperforming original baselines in Task Outcome (82.95% and 46.61%, respectively). Controlled studies examine how skill instructions, training signals, and contract-state feedback affect both metrics, while withholding interventions evaluate behavioral dependence on observation content.

Sun 4 OctArtificial Intelligence
The gist
Following instructions usually means just getting the final answer right, but the authors explore how to get computers to follow detailed step-by-step rules for tasks, like a recipe. They create a system called Grounded Skill-Following that breaks tasks into phases and checks each step is done correctly as the computer works. This system guides the computer throughout the task, giving feedback and rewards as it completes important steps in order. Their approach shows high rates of correctly completed procedures, even surpassing some original methods on task outcomes.
Open → 2610.05161v1

Strata improves real-time strategy game agents with role-based learning

STRATA: Self-Learning Through Role-Aligned Tiered Agents for Real-Time Strategy Games

Abstract: Real-time strategy (RTS) games require agents to coordinate economic development, production and construction, base defense, unit organization, and attack timing over long matches. Existing studies have applied large language models to command decision-making in RTS games, enabling agents to read textual game states and generate high-level plans. However, long inference latency can cause them to miss critical tactical events. The complexity and tactical diversity of full RTS matches also leave existing systems heavily dependent on manually written experience-based prompts, with limited ability to learn continuously from past games. We present STRATA, a role-aligned hierarchical system with cross-game self-learning for Red Alert. STRATA assigns in-game strategic, logistical, and tactical decisions to a Strategic Agent (SA), Logistics Agent (LA), and Tactical Agent (TA), respectively. The SA generates high-level directives based on the global game state and relevant experience cards, while the LA and TA handle logistics and tactical execution. After each match, a Review Agent (RA) derives candidate experience from game traces, validates and revises it using evidence from subsequent matches, and compresses strategic experience supported across multiple games into concise experience cards for SA retrieval. We evaluate STRATA through the formation of experience cards, full-match comparisons before and after learning, and experience learning against AI opponents with different play styles. Under a fixed scenario, using the learned experience cards increases the observed win rate from 30% to 100%. Sequential learning against AI opponents with different play styles also produces distinct long-term strategic experience.

Wed 30 SeptArtificial Intelligence
The gist
Real-time strategy games are complex and need players to manage long-term plans and quick tactics at the same time. The authors present STRATA, a system that separates decision-making into strategic, logistical, and tactical agents to better handle these challenges. STRATA learns from past games by reviewing them and creating experience cards, which help improve future match performance. Using these learned experiences, STRATA raised its win rate from 30% to 100% in tested scenarios.
Open → 2609.38881v1

Multi-agent systems improve task coordination with coherent memory and roles

Topological Coherence for Self-evolving Multi-agent Systems

Abstract: Complex tasks inherently couple workflow structure, agent responsibility, collaboration, and memory access: task regions delimit responsibility and tool scope, cross-region dependencies give rise to handoffs, and ownership boundaries delimit private and selectively shared memory. Existing methods can jointly optimize agent and communication structures, yet such optimization does not by itself require responsibility, handoff, and memory boundaries to remain consistent with task dependencies. We term this requirement topological coherence. We introduce TOCOMAS, a Topology-Coherent Multi-Agent System. TOCOMAS grounds a task graph in tool interfaces, organizes compatible task nodes into reusable responsibility domains, and derives dependency-induced and profile-conditioned collaboration together with boundary-regulated memory visibility. During online self-evolution, TOCOMAS proposes coupled changes to agent, collaboration, and memory policies, retaining for subsequent tasks only candidates that satisfy structural constraints and improve evaluated reward. Across BBEH, WorkBench, SWE-Bench-Verified, and CoMemBench, TOCOMAS improves task success over baselines across backbones. CoMemBench also shows gains over the self-evolving baseline in verified progress, handoffs, and memory isolation.

Tue 29 SeptArtificial Intelligence
The gist
Coordinating many software agents to work together on complex tasks requires clear roles, memory sharing rules, and handoffs between parts of the task. The authors highlight a concept called topological coherence, ensuring these boundaries fit the task's structure. They created TOCOMAS, a system that organizes agents and memory based on task dependencies and evolves these setups during use, improving success on benchmark tests. This helps multi-agent systems work more reliably and efficiently over time.
Open → 2609.37953v1

Reinforcement learning improves by filtering bad training paths

Not All Rollouts Are Worth Learning: On Trajectory Valuation for Post-Training Reinforcement Learning

Abstract: We consider the problem of trajectory valuation in reinforcement learning: how to identify and mitigate detrimental trajectories during online training. Unlike classification, where data valuation relies on fixed training and validation sets, reinforcement learning involves dynamically generated trajectories without explicit validation signals, making conventional influence-based methods inapplicable. We propose Dynamic Trajectory Valuation (DTV), a simple and efficient framework that estimates trajectory utility at the mini-batch level and filters detrimental trajectories based solely on gradient information. By operating at the optimization level, DTV integrates seamlessly with existing reinforcement learning pipelines with minimal overhead. Extensive experiments across diverse settings, including PPO, GRPO, and DPO, demonstrate that DTV consistently improves performance, enhances data efficiency, and stabilizes optimization.

Mon 28 SeptMachine Learning
The gist
Training AI to make decisions often involves learning from examples called trajectories, but some of these examples can actually hurt learning. The authors show that existing methods to value data don't work well for reinforcement learning because the data is generated on the fly and there's no straightforward way to tell which examples are good or bad. They introduce a new method, Dynamic Trajectory Valuation (DTV), which uses the training process itself to figure out which training paths to keep or ignore, helping the AI learn better and more efficiently. Tests with popular reinforcement learning algorithms show that this approach makes training more stable and improves performance.
Open → 2609.35072v1

Posterior sampling achieves optimal exploration rates in reinforcement learning

Minimax-Optimality of Posterior Sampling for Reinforcement Learning

Abstract: Posterior sampling for reinforcement learning (PSRL) is one of the simplest and most effective exploration methods, but a basic question has remained open: does unmodified PSRL achieve minimax regret without structural assumptions on the prior? We answer yes. Exact vanilla PSRL is minimax optimal in leading-order Bayesian regret under arbitrary correlated priors. The difficulty is that a posterior-sampled transition model is coupled with its own continuation value. We overcome this with a common empirical transition reference that isolates the resulting value mismatch and a Bellman-based variance argument that controls it without an extra leading-order state-space factor. For finite-horizon, time-inhomogeneous tabular MDPs with unknown stochastic rewards, this yields the minimax $\widetilde{O}(\sqrt{SAH^3K})$ regret rate under arbitrary joint priors over rewards and transitions. The same proof principle gives the minimax $\widetilde{O}(d\sqrt{H^3K})$ rate for linear-mixture MDPs under arbitrary joint parameter priors.

Sun 27 SeptMachine Learning
The gist
Reinforcement learning helps computers learn to make decisions through trial and error. One popular method, posterior sampling, was simple but it wasn’t clear if it worked as well as possible in the most general situations. The authors show that this method actually does achieve the best possible performance in terms of learning speed and decision quality, even with complex prior knowledge. They used mathematical tools to carefully analyze why it works so well, confirming its effectiveness without extra assumptions.
Open → 2609.33246v1

Robot maps broken down into scenes to find changes fast

Online Geometric Change Detection via Scene Decomposition

Abstract: Autonomous robots are increasingly deployed on long duration single- and multi-session missions in dynamic environments, where the ability to identify environmental changes such as fallen trees or opened doors provides important contextual information for online planning. We propose a framework called Change Detection via Scene Decomposition (CDSD) for accurate online geometric change detection using LiDAR or RGB-D sensors. Recent advances in geometric SLAM have made it possible to generate dense, tightly aligned maps without post processing, but comparing global maps across entire sessions is computationally expensive and does not allow for single-session online change detection. CDSD instead spatially decomposes mapped environments into unique scenes where changes can be found efficiently by comparing dense, local subsets of the global map called submaps. As the first submap-based approach for geometric change detection, we identify and address the following core challenges: 1) identifying appropriate scenes for change detection that require minimal redundant information; 2) generating dense and representative submaps for each scene; 3) detecting changes between submaps with differing fields of view; and 4) processing detected changes for real-time map reconstruction. Results demonstrate our algorithm on custom datasets collected at the Army Research Laboratory facility in Graces Quarters, Maryland, and on open-source multi-session change detection datasets.

Tue 15 SeptRobotics
The gist
Finding changes in environments helps robots understand what’s around them and plan better as things change. The authors present a way to break big robot maps into smaller scenes, called submaps, to quickly spot differences like doors opening or trees falling. This method uses detailed 3D data from sensors and works while the robot is moving, instead of waiting until after a long mapping session. It’s tested on real and open datasets, showing it can detect changes efficiently and update the map in real time.
Open → 2609.17302v1