Papers for

robotic software engineers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

World models improve agent success in complex tasks by predicting outcomes

Learning to Act with World Models by Valuing What Comes Next

Abstract: A sparse terminal reward tells an agent little about its steps. Much of the missing information is implicit in the environment's dynamics, because each state transition can change the conditions for success long before the reward arrives. A world model trained on real transitions can learn to predict what each action changes, but a predicted observation does not by itself show whether success has become more likely. With a binary terminal reward, PPO's critic makes this judgment by estimating the current policy's probability of success, but only for states the agent reaches. Our method, WM-PPO, elegantly brings the two together. For each action the policy samples, executed or not, the world model predicts the next observation and the critic values the predicted successor. In the policy channel, comparisons among these values give the policy auxiliary advantages; in the value channel, their mean gives the critic an auxiliary target. A shared gate weights both channels according to how reliably comparisons with similar margins predicted real outcomes earlier in training. The policy, critic and world model all learn from the agent's own interaction, with no teacher or privileged information, and only the policy is used at inference. With a comparable number of real environment steps, WM-PPO raises success from PPO's 63.5% to 87.5% on ALFWorld and from 39.3% to 52.9% on ScienceWorld. It matches the strongest PPO refinements we tested on ALFWorld and exceeds the best of them by 6.5 points on ScienceWorld. The trained policy more often completes prerequisite steps before placing an object or committing to an answer.

Mon 9 NovArtificial Intelligence
The gist
When a computer learns to solve a task but only finds out if it succeeded at the end, it's hard to know which steps helped. The authors combined a method that predicts what happens next with one that judges how good each step is. Their approach lets the computer look ahead and decide better which actions lead to success. This helped agents do much better at navigating and completing complicated tasks in virtual environments. The new method learns entirely from experience without extra hints and only uses what it learned to choose actions.
Open → 2611.12200v1

Llm planning improves efficiency with expectation observation checks

As Expected? Efficient LLM Planning via Expectation-Observation Consistency Verification

Abstract: LLM planning often relies on reactive executors (e.g., ReAct) that generate a single action at each step to remain grounded in the latest observations, incurring substantial inference cost. Batched executors reduce this cost by predicting multiple actions in a single model invocation, yet they are prone to compounding errors, as intermediate observations are ignored. We propose an expectation-observation consistency verification (EOCV) mechanism for batched executors, preserving grounding while improving efficiency. The mechanism generates expectations of action outcomes and compares them with the resulting observations. Consistency permits continued execution, whereas mismatches trigger early termination and replanning from the current observation. Experiments on text-based and multimodal benchmarks across multiple backbones show broadly comparable average task performance between EOCV and reactive executors, with a 29-35% reduction in average weighted token usage. Moreover, the mechanism is model-agnostic and remains effective when integrated into state-of-the-art planners.

Mon 2 NovComputation and Language
The gist
Planning with large language models (LLMs) often needs many back-and-forth steps that can be slow and costly. The authors introduce a method called expectation-observation consistency verification (EOCV) that lets the model plan multiple actions at once but checks whether outcomes match its expectations to avoid big mistakes. If something doesn’t match, the model stops and replans using the latest information. This approach keeps performance similar to step-by-step methods but uses fewer computing resources.
Open → 2611.02438v1

AI agents not instantly useful for industrial robot programming

Can AI Agents Help Robotic Engineers? Lessons from Two Focus Days in Industry

Abstract: Software developers increasingly use AI agents that execute multi-step programming tasks on their own. However, it is unclear if and how this transfers to robotic engineering, where engineers program industrial robots in niche, vendor-specific languages and rely on visual feedback from simulation tools. Within this paper, we report a case study on the use of AI agents by an SME that programs industrial robots. Our results show that the agent did not make the engineers more productive, with the exception of a task that required the generation of highly repetitive code. However, the results with the agent improved over time, as the engineers learned how to use it and the agent was adapted to the domain. Moreover, the experience with AI mattered more for the success with the agent than the experience with robotic engineering. This shows that AI agents are not a plug-and-play solution. Instead, companies must train their engineers on the usage of agents, plan time for a learning curve, and ensure that agent configurations match the domain and developer requirements. The strong learning curve we observed further implies that studies on productivity gains with AI should be longitudinal, as they otherwise miss how engineers adopt agents in practice.

Fri 23 OctSoftware EngineeringArtificial Intelligence
The gist
Using AI agents that can write code on their own works well for some software developers, but robotic engineers program robots using special languages and tools, which makes it different. The authors studied a small company that codes industrial robots using AI agents, and found that the AI helped mainly with repetitive coding tasks, but didn’t make the engineers faster overall. However, the engineers got better at using the AI over time, and the AI improved when adapted to the robot programming domain. The study shows that companies need to invest effort and time to train engineers and customize AI before it becomes truly productive.
Open → 2610.32636v1

Agents improve decisions by predicting future with learned experience

Self-Improving Agents with World Model Predictive Paths

Abstract: Large language model (LLM) agents are increasingly expected to interact with environments and improve their decisions during deployment. However, how anticipating future outcomes and drawing on accumulated interaction experience can jointly support decision-making remains underexplored. We propose a test-time framework, inspired by Model Predictive Path Integral (MPPI) control, that combines multi-branch predictive analysis with experience-based adaptation. At each step, the agent proposes multiple candidate actions and imagines their outcomes with a world model, so that alternatives can be compared without being executed in the environment. Since language environments provide neither explicit dynamics nor costs, the predicted outcomes are evaluated for task progress and risk, and MPPI-style weights over the resulting costs guide the final decision. A dual-branch experience memory enables self-improvement: the planner branch refines candidate generation and action selection, while the world model branch improves next-observation prediction. Experiments on ALFWorld and ScienceWorld across three backbones demonstrate the strong overall performance of our method and the effectiveness of experience-guided predictive control. Our code will be made publicly available.

Tue 20 OctArtificial Intelligence
The gist
Making smart decisions in changing environments is hard because it needs predicting future effects and learning from past experiences. The authors created a method where an AI agent imagines different possible moves using a model of the world to see what could happen without actually trying each one. It then picks the best choice by weighing how well it advances the goal and avoids risks, improving itself with experience over time. They tested this on language-based tasks that don’t have clear rules or costs, showing better decision-making by combining prediction with learning from past actions.
Open → 2610.28945v1

Flow-based method boosts smoothness and speed in robot control

Adaptive Mean Flow for Responsive Closed-Loop Robot Control

Abstract: Diffusion- and flow-based robot policies have recently become widespread in robotic Imitation Learning (IL) due to their high performance and ability to model continuous and multimodal distributions. However, the iterative denoising procedure used by these models introduces significant prediction latency, hindering high-frequency closed-loop robot control and leading to jittery, unstable motion when frequent updates to the robot's action predictions are used. Therefore, it is common practice to train models to predict chunks of actions that can be executed sequentially without feedback, even when this reduces responsiveness and may mean the most recent state information is not used. In this article, we present Adaptive Mean Flow (AMF), a flow-based IL method that enables smooth and responsive, fully closed-loop robot control. AMF uses Mean Flow, which is an accelerated form of Flow Matching (FM), to minimize prediction latency. To ensure smoothness and consistency across predictions, AMF uses a corrupted version of the trajectory from the previous step when predicting new robot actions, with the signal-to-noise ratio increasing over the time parameter of the trajectory. This discourages large changes in the prediction from one step to the next, while allowing freedom to adapt the predictions for future steps. We evaluate AMF across a wide range of simulated and real robot tasks and demonstrate significantly improved performance compared with baselines. Code: https://github.com/akselva/Adaptive-mean-flow-RoboticIL.

Mon 5 OctRoboticsArtificial Intelligence
The gist
Robot controllers that predict future actions often face delays making movements jerky and unstable. The authors introduce Adaptive Mean Flow (AMF), a method that speeds up prediction while keeping actions smooth and responsive. This approach uses a trick where it slightly corrupts previous planned actions so new predictions don’t jump around too much. AMF was tested on many robot tasks and showed better performance than previous methods.
Open → 2610.06089v1

Agents reuse historical action credit to reduce tool interactions

When Does Action Credit Need Updating?

Abstract: Tool-using agents are continually updated with new interaction data. After each policy update, however, previously estimated action credits may become stale. Recomputing them from scratch can require many additional tool calls and environment interactions, making repeated updates increasingly expensive. We ask a simple question: when does historical action credit actually need to be updated? Our key observation is that a change in action value does not necessarily imply a change in the decision. Historical credit can still be useful as long as policy-induced drift is too small to overturn the existing action ranking. Building on this idea, we introduce pairwise branch sensitivity to capture how strongly a policy update affects the downstream regions that distinguish two candidate actions. We then derive a first-order anchored credit-transport estimator that updates historical credit using old interventional trajectories, and propose a Decision-Sufficient Credit Gate (DSC-Gate) that chooses whether to reuse, transport, or resample credit. Experiments show that branch sensitivity explains credit drift substantially better than global policy distance. With sufficient historical data, credit transport reduces estimation error, while its benefit to decision making is concentrated on updates that affect action-distinguishing branches. On a fully independent test set, DSC-Gate changes mean regret by only +0.00004 relative to a gap-based gate while reducing mean new tool steps from 472 to 286, a 39.4% reduction. We observe the same pattern after a real tool-agent parameter update. Overall, our results show that agents do not need to recompute action credit after every policy update: much of the historical evidence can be reused or cheaply corrected, reducing the additional interaction required to keep action decisions up to date.

Thu 24 SeptArtificial Intelligence
The gist
For agents that use tools, updating decisions after each change can be slow and costly if they recalculate old information every time. The authors found that agents often do not need to redo all their previous assessments because small updates usually do not change which option is best. They created a way to measure when old information stays useful and a method to update it efficiently. Their system cuts down the number of times agents have to redo tool actions by nearly 40% while keeping decision quality nearly the same.
Open → 2609.29007v1

IncentRL balances guidance and task success in reinforcement learning

IncentRL: The Trade-Off Between Preference Guidance and Task Performance

Abstract: Preference-based reward shaping can guide reinforcement learning, but adding preference signals to the reward may unintentionally change the task being optimized. We address this problem with IncentRL, a framework that introduces preference guidance while explicitly characterizing its effect on external-task performance. IncentRL adds a Kullback--Leibler (KL) penalty between a specified outcome distribution and a preferred distribution. For finite discounted Markov decision processes with bounded shaping costs, we derive an external-value perturbation bound, establish a sufficient strict-action-gap condition for preserving the original optimal policy, and characterize the large-weight regime through discounted cumulative preference cost. Exact examples clarify the limits of these guarantees, including tied optima and support mismatch. We study a practical implementation using a hand-designed, distance-based outcome proxy, a fixed preference distribution, and score-weighted coefficient search. On MiniGrid DoorKey-8x8, the reported three-seed mean success rate after two million training steps reaches 98\% with coefficient 0.01, compared with 90.5\% for the reported zero-coefficient baseline, while the search progressively shifts toward smaller coefficients. Together, these results provide a principled view of the central trade-off in preference-based RL: using additional guidance to improve learning without excessively distorting the original task objective. The current experiments remain descriptive and do not yet isolate KL shaping from simpler alternatives.

Fri 18 SeptMachine Learning
The gist
Reinforcement learning often uses extra signals to guide training, but these signals can accidentally change what the system aims to do. The authors introduce IncentRL, a method that adds guidance while measuring how much it changes the original goal. Their math shows when the original goal stays safe and how big guidance signals affect results. They test their idea on a simple game, improving success rates by carefully choosing how strong the guidance is. This work highlights how to use extra hints without messing up what the system is supposed to learn.
Open → 2609.21525v1