Papers for

robot software developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Robot software issue fixes validated by running tests across multiple domains

Robo-SWE: An Executable Issue-Resolution Dataset Mined from Robot Software Repositories

Abstract: Coding agents are evaluated mostly on general-purpose software, but domain-specific systems require domain-aware evaluation. Robot software is a particularly challenging case: Robot Operating System 2 (ROS 2) packages have their own build tools, simulators and learning frameworks are built on physics engines, and behavior that depends on timing and hardware is hard to test. We present Robo-SWE, a dataset of 236 executable issue-resolution tasks mined from merged pull requests in 28 public robotics repositories in four domains: ROS 2 systems, simulation and physics engines, robotics algorithm libraries, and robot-learning frameworks. Each task contains the original issue text, a container recipe for the repository at the commit before the fix, the developers' fix, and the developers' tests, which fail before the fix and pass after it in repeated offline runs. The dataset also keeps a provenance ledger of all 1,567 candidate pull requests, with the stage each reached and the reason for every rejection, and, as a first use, 3,540 graded attempts by five coding-agent setups with recorded patches and 3,529 available trajectories. Across three runs per primary setup, resolve rates range from 45.8% for Claude Sonnet 5.5 to 56.5% for Claude Opus 5.5; GPT-6.1 Sol reaches 54.1% at maximum effort and 52.5% at medium effort, and GPT-6 Astra 51.4%. GLM-5.3 resolves 46.2%, and Kimi K3.1 Pro 43.9%. The tasks are validated by execution; verification of their fairness by human reviewers is future work. The dataset and public leaderboard are available online.

Wed 11 NovSoftware EngineeringRobotics
The gist
Fixing problems in robot software is hard because robots rely on special tools and hardware that are tricky to test. The authors collected a large set of real robot software issues along with the fixes and tests that show the fixes work when run. They tested several coding agents on these robot-related tasks to see how well the agents could solve the issues. This dataset can help improve how robot software is tested and how automated tools fix problems in this specialized area.
Open → 2611.16116v1

Grasp feasibility predictors fail when obstacles change approach paths

Auditing Grasp Feasibility Predictors with Planner-Verified Counterfactuals

Abstract: Grasp-feasibility predictors report near-perfect accuracy on Task and Motion Planning benchmarks, but high accuracy on these benchmarks does not verify that a predictor uses the collision geometry that the motion planner checks. We propose a planner-verified counterfactual (CF) test: holding the grasp pose fixed, we insert a solid obstacle into the approach corridor, add its points to the model input, and let a planner re-derive the label, which flips from feasible to infeasible. Across eight predictors spanning point clouds, voxels, depth, and object tokens, CF-pair AUROC falls from high natural AUROC (0.920-0.996) to chance (0.470-0.504), and the flip holds across four planners, two arms, and a real-object obstacle. Predictors fail at different resolutions, from a bounding box to a 12.5 cm voxel. In the point-cloud models we dissect, the obstacle either moves the representation in a direction unrelated to whether it blocks the grasp or is not registered at all. A clearance-aware read of the corridor points, trained on the unmodified benchmark with no counterfactual scenes, reaches 0.983 CF-pair AUROC, but it remains bound to its trained obstacle placements; a placement-invariant clearance representation is still open. We release the counterfactual labels, metrics, datasheet, and relabeling pipeline at https://github.com/codewithsajid/graspfc-cf.

Fri 6 NovRobotics
The gist
Robots use grasp feasibility predictors to decide if a hand position can safely pick up an object. This paper shows that these predictors often ignore important obstacles by testing with new, inserted obstacles and checking with motion planners. The authors find that predictor accuracy drops to chance when obstacles block the grasp path, meaning the models don't truly understand collision risks. They propose a new testing method and offer improved predictor designs that better consider obstacle clearance.
Open → 2611.09130v1

PrimeSim simulates robot actions with consistent multiview long-term accuracy

PrimeSim: A Generalist World Model Simulator from Large-Scale Cross-Embodiment Demonstrations

Abstract: A robot world model that serves as a simulator must do more than generate plausible videos: it must faithfully follow commanded actions, maintain consistency across camera views, and sustain coherent long-horizon rollouts. We present PrimeSim, an action-conditioned, multiview world model simulator that addresses these challenges through progressive midtraining and posttraining. Starting from a pretrained video generator, we midtrain on 30,110 hours of robot demonstrations spanning 22 platforms. To improve action following, we introduce a unified cross-embodiment action representation and frame-aligned dual-path conditioning to strengthen action-frame correspondence. To maintain multiview consistency, we develop view-aware interleaved attention with explicit camera identities, enabling cross-view information exchange while preserving view-specific representations. We further introduce a fast-video, slow-memory posttraining framework that transforms the bidirectional video generator into an efficient autoregressive simulator. By combining fast causal generation, self-rollout training, and hierarchical temporal memory, PrimeSim mitigates error accumulation from self-generated histories and supports coherent long-horizon simulation. PrimeSim runs at 10.1 FPS, remains stable over long-horizon rollouts on rigid and deformable manipulation tasks, and transfers to held-out platforms within 100 to 200 gradient steps where the pretrained backbone does not. Against five open-source action-conditioned world models it leads four of the five WorldSimProbe suites and posts the best overall EWMBench score, while fine-grained local calibration still trails the field; its rollouts are faithful enough to support GRPO-based policy optimization. We release the model weights, inference code, and training code.

Tue 3 NovRobotics
The gist
Simulating how robots act in different scenes is tricky because it requires following commands precisely, keeping all camera views consistent, and staying accurate over long periods. The authors created PrimeSim, a model that learns from a huge variety of robot videos across many types of machines and improves how it matches actions with visual outcomes. It also shares information between different camera angles to keep consistency and uses a special training method to reduce errors when predicting future frames. PrimeSim can run simulations fast, last a long time without losing accuracy, and quickly adapt to new robots it hasn't seen before.
Open → 2611.04870v1

Tactile sensing improves robot touch tasks across varied conditions

TacMind: Benchmarking and Modeling Tactile Interaction States for Robotic Manipulation

Abstract: Tactile sensing provides robots with direct observations of contact physics, local geometry, and interaction dynamics, making it a vital source of feedback for contact-rich manipulation. However, existing tactile manipulation studies are often developed and evaluated within specific task settings, making it difficult to systematically compare how tactile information contributes under different interaction conditions. In this work, we develop a benchmark in simulation comprising 128 tasks organized around three complementary evaluation regimes: Precise, Recognition, and Dynamic. The benchmark further includes controlled static and dynamic task variants and 12 perturbation settings for evaluating robustness under changes in visual and physical conditions. These settings motivate studying tactile information as a temporally evolving interaction representation rather than only as an instantaneous observation. We therefore propose an action-conditioned latent predictive world model that integrates tactile history and learns a tactile state through prediction of future tactile latents. Building on this representation, we design a model that conditions action generation on the learned tactile state and uses predicted future tactile information to refine candidate actions through predictive tactile reflection. Experiments on the proposed benchmark show consistent gains from incorporating tactile information across the evaluated regimes. Project Page: https://tacmind-bench.github.io/

Fri 30 OctRobotics
The gist
Robots often need to touch and handle things carefully, but it's hard to compare how well their sense of touch works in different tasks. The authors created a large set of robot touch tasks in simulation to test this. They also developed a method that helps robots remember and predict what their sense of touch will feel like in the future, which improves how they decide to act. Their tests showed that using touch information consistently helped robots perform better on these tasks.
Open → 2611.00455v1

Vision guided robot navigation learns to plan and act together

MATCH: Execution-Aware Hierarchical Reinforcement Learning for Vision-and-Language Navigation in Continuous Environment

Abstract: Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires instruction-consistent planning and reliable execution under collision constraints. Yet many hierarchical approaches optimize the planner while keeping execution fixed. We introduce MATCH, a framework that jointly learns a language-conditioned planner and a goal-conditioned primitive-action executor. An execution-aware topological graph represents start-dependent execution options and integrates executor value estimates with realized outcome memory, enabling the planner to select execution requests and refine their targets through bounded continuous offsets. A transition-aware dense macro reward balances ordered progress along the reference route against execution residual, encouraging executable requests without substituting local success for instruction following. A staged curriculum culminates in synchronized on-policy updates, with the executor learning on planner-generated requests and the planner adapting its requests to the executor's current behavior. With 0.5B parameters, MATCH achieves state-of-the-art performance on the R2R-CE and RxR-CE benchmarks. Preliminary real-robot deployments further demonstrate its practical potential.

Wed 28 OctRobotics
The gist
Robots that navigate by following spoken or written instructions need to both plan routes and avoid bumps or obstacles while moving. This paper presents MATCH, a method that trains a robot’s planning and movement parts together, so the robot can better understand instructions and move more safely and efficiently. The authors show that MATCH works well in simulated tests and even in experiments with real robots, proving its promise for practical use.
Open → 2610.38248v1

Humanoid robots smoothly connect motions using human movement patterns

Robot Motion Transition with Human Motion Prior

Abstract: Humanoid motion-tracking policies can reproduce diverse behaviors, yet connecting independently acquired motions into a continuous sequence remains challenging. Interpolation connects boundary poses without considering whether their relative position, heading, and timing provide suitable conditions for a transition. A generative motion prior can produce richer motion, but poorly arranged boundaries may cause it to generate excessive locomotion, turning, or another unintended action. We present Spatio-Temporal Alignment with a Motion Prior (STAMP), a framework for connecting independently acquired humanoid motions through a Robot-to-Human-to-Robot process. STAMP uses human motion space to draw on rich motion priors and reuse transition knowledge across robot embodiments. Its key component is a learned alignment model that determines the boundary conditions for motion inbetweening. Given the two motions retargeted to a human rig, the alignment model ranks candidate positions, headings, and transition durations for the following motion. A human motion prior generates the transition under the selected alignment. The completed sequence is retargeted to the robot and used as a reference to train a motion-tracking policy. We evaluate the tracking success of the generated transitions on Unitree~G1 and~ROBOTIS K1. Physical robot demonstrations further show that STAMP connects independently acquired motions into long-horizon sequences. Project page: https://childtoy.github.io/STAMP-project

Thu 22 OctRobotics
The gist
Making humanoid robots perform many different moves in a row is hard because simply joining the end of one move to the start of the next can look awkward or wrong. The authors developed STAMP, which uses knowledge learned from human motions to find the best way to align and blend these moves smoothly. By translating robot motions to a shared human motion format, they decide how to position, angle, and time transitions before turning them back into robot actions. They tested this on real robots and showed it helps create long, continuous motion sequences.
Open → 2610.30817v1

Physically grounded world model improves quadruped robot action predictions

From Pixels to Executable Physics: Behavior-Grounded World Models for Embodied Action Consequence Prediction

Abstract: In outdoor environments, varying terrain and contact conditions can cause identical action commands to produce different motion outcomes for quadruped robots. Predicting these outcomes is essential for reliable navigation and informed path selection. Existing video world models can generate visually plausible futures, yet often struggle to capture robot dynamics and terrain interactions, leading to overly optimistic predictions of action outcomes. We present DR-WM, a physically consistent world model that incorporates robot-terrain contact dynamics to predict the consequences of quadruped actions. Starting from an initial RGB image, our method reconstructs scene geometry and integrates dense, dimensionless priors over terrain contact properties, calibrated using real-world action-response data. The resulting terrain model supports the simulation of physics with spatially varying contact properties such as friction. Given the initial state of the robot and candidate action sequences, we simulate its future motion and use the resulting camera trajectories and rendered geometry to guide the training of a video world model. The generated future observations are then used to update the terrain reconstruction, enabling iterative model rollouts without additional real-world images. Extensive experiments demonstrate that DR-WM substantially outperforms geometry-only baselines and baselines with uniform physical parameters on key physical metrics characterizing action outcomes, while achieving the best performance among the evaluated baselines. These physically grounded predictions support more informed action selection: across 90 challenging scenarios, an 8B vision-language model augmented with DR-WM outperforms an unassisted 32B vision-language model on five of six evaluation metrics.

Tue 13 OctRoboticsComputer Vision and Pattern Recognition
The gist
Quadruped robots walking outside can experience different results from the same commands because the ground varies a lot. The authors present a method called DR-WM that uses pictures to build a physical model of the terrain and simulate how the robot will move on it. This helps predict what will actually happen when the robot takes certain actions, making the predictions more accurate than previous methods that did not consider detailed physical terrain properties. Their approach improves decision-making for robot navigation in challenging environments.
Open → 2610.17630v1

Large language models generate reactive robot behavior scripts from vague instructions

Declarative Reactive Robot Behaviors from Imperative Language Models

Abstract: Large Language Models (LLMs) are predominantly trained on imperative programming languages, and existing work on LLM-based robot programming reflects this: it focuses on goal-directed tasks expressed as plans. However, many robotic applications require not tasks but behaviors--open-ended, continuous patterns of sensory-motor engagement with no termination condition (e.g., greeting visitors, patrolling an area). Such behaviors are better served by declarative and reactive programming, on which LLMs have virtually no training. We investigate whether this gap can be bridged. Using Playful, a reactive scripting language, we show that an LLM-based coding agent can generate correct declarative and reactive programs for robotic behaviors. These programs are produced from under-specified natural language descriptions, given suitable context and a set of reusable programming guidelines. The guidelines encode design principles that redirect the agent from imperative defaults toward proper reactive patterns. Developed once by an expert, they generalize across robots and behaviors. Starting from only the generic robot operating system API, the agent jointly designs both the imperative component (Python modules for sensing, acting, and evaluating) and the declarative component (the reactive script that composes them into behavior logic). The declarative nature of reactive programming supports both incremental refinement through conversational turns and one-shot generation of complete behaviors.

Mon 12 OctRobotics
The gist
Robots often need to perform ongoing behaviors like greeting visitors or patrolling, which don’t have clear end points and are different from goal-based tasks. The paper shows that large language models, usually trained to write step-by-step instructions, can also create reactive robot programs that handle continuous behaviors. The authors use guidelines to help the model write these kinds of programs in a reactive language that works well for ongoing actions. This approach lets robots carry out complex behavior patterns from simple, natural language descriptions and can be refined over conversations or generated all at once.
Open → 2610.17083v1

Endoscopic robot policies improve through physical experiment feedback

EndoRSI: Physical-Evidence-Driven Recursive Improvement of Endoscopic Robot Policies

Abstract: Artificial intelligence is beginning to automate scientific discovery, yet physical robot-policy research still relies on scientists and engineers to interpret experimental failures and decide how controllers should be revised. We present EndoRSI (Endoscopic Recursive Self-Improvement), a human-governed, self-improving robotic research system in which the embedded EndoForge coding agent iteratively develops and refines endoscopic robot policies using evidence from physical experiments. Physical execution remains isolated within a safety-constrained runtime under human authorisation. In motorised endoscopy, validation evidence first caused the agent to reject an initial action representation, while subsequent physical experiments falsified an offline-supported insertion strategy by revealing hazardous closed-loop behaviour absent from the source data. Failed physical rollouts were then incorporated into policy revision; insertion success subsequently increased from 2/21 in the first generation to 17/21 in the fourth. The frozen final policy retained competence across visual and hardware shifts and was the best performing autonomous policy across six independent ex-vivo porcine specimens. These results show that physical robot interaction can serve as experimental evidence for agent-driven robot policy research.

Fri 9 OctRobotics
The gist
Controlling robots inside the body, like those used for endoscopy, is tricky and needs constant adjustments. The authors designed a system called EndoRSI where a robot’s control programs get better by learning from real physical tests. The system safely runs experiments to see what works and what doesn’t, then changes its approach accordingly. Over four rounds of testing, the robot got much better at inserting itself without causing harm. This helps robots learn from the real world, not just simulations.
Open → 2610.13376v1

Latent world action model improves humanoid robot control

Being-M0.7: A Latent World-Action Model for Humanoid Robots

Abstract: Humanoid loco-manipulation requires coordinated locomotion and manipulation informed by future scene evolution and whole-body motion, yet learning these capabilities is constrained by scarce robot demonstrations. Human video and motion datasets offer scalable supervision, but many contain only video or motion rather than paired video-motion data. Moreover, human motion does not directly specify executable robot actions. We present Being-M0.7, a latent world-action model that transfers visual-motion priors learned from mixed-modality human data to humanoid control through pre-training, robot mid-training, and action post-training. We curate a corpus from more than 10,000 hours of raw human-centric data, integrating video-only, motion-only, and paired video-motion streams to learn complementary visual dynamics and whole-body kinematic structure. Joint prediction of future latent visual states and motion encourages visual representations to encode future kinematics. Robot mid-training adapts this coarse-grained prior to robot viewpoints and body dynamics. During action post-training, an action expert combines visual predictive representations from the frozen, adapted prior with current images and proprioception through gated cross-attention, grounding predictive context in executable whole-body commands. Being-M0.7 achieves the highest aggregate success rate among the compared baselines on SIMPLE and matches the strongest baseline on real-world Unitree G1 loco-manipulation tasks.

Thu 8 OctRoboticsComputer Vision and Pattern RecognitionMachine Learning
The gist
Humanoid robots need to move and manipulate objects smoothly, but teaching them is hard because there aren’t many robot demonstrations. The authors made a system called Being-M0.7 that learns from lots of human videos and motion data, even when these datasets don’t match perfectly. This system then adapts what it learned to control robots, helping them predict how the world and their own bodies will move. Their approach improves success in robot tasks involving walking and manipulating objects.
Open → 2610.11283v1

Legged robots adapt quickly to changes using GPU-powered simulation models

LLA-MPPI: Rapidly Adaptive Whole-body Control of Legged Robots with GPU-Accelerated Parallel Simulations

Abstract: Real-time whole-body controllers for legged robots typically plan through a fixed nominal model and degrade when the deployed dynamics change. Adaptive methods typically require a model structure that contact dynamics do not provide, or they need offline training for each anticipated condition. We present Look-back and Look-ahead Adaptive Model Predictive Path Integral control (LLA-MPPI). The method converts whole-body adaptation into selection over a bank of GPU-batched contact simulators with different physical or structural parameters. Windowed prediction errors select the simulator that best explains recent motion. A whole-body MPPI planner optimizes controls through the selected model. The framework requires no offline training, and its selected hypotheses are physically interpretable. Across four simulated tasks, it achieves 97.5% success while the strongest baseline reaches 74% and an oracle with the true model reaches 98.5%. Hardware validation on a Unitree Go2 shows the robot walking under a payload added mid-run, walking after one leg is disabled, and pushing a box to its goal while increasing its mass on the fly. Code, videos, and project details are available at: https://lla-control.github.io

Wed 7 OctRobotics
The gist
Legged robots often struggle when their environment or body changes because their control systems rely on fixed models. The authors created a method that lets robots try out many different simulations in parallel, using powerful GPUs, to find the one that best matches their recent movements. This helps the robots adjust their actions in real time without needing prior training for every situation. Their method was tested in simulations and on a real robot that could walk with a heavy load, a broken leg, and push objects with changing weight.
Open → 2610.10465v1

Robot behavior fails when its description changes despite physical sameness

The Robot Is Not Its Description: GaugeBench for Representation Robustness in Morphology-Aware Policies

Abstract: A robot description does more than specify a physical mechanism: it also encodes arbitrary conventions, such as joint-axis direction, joint-angle zero, and the order and names of links and joints. Morphology-aware policies consume interfaces built from these descriptions, yet cross-embodiment evaluation typically changes the robot while keeping those conventions fixed. This leaves a simple question unanswered: does behavior survive when the robot stays fixed but its description changes? GaugeBench isolates this case by rewriting a fixed mechanism under physically equivalent conventions, verifying that its physics and policy interface are preserved, and then evaluating the same policy weights. The result is stark: three MetaMorph policies score 4030.6 on 80 familiar robots, but only 51.6 when those same robots are equivalently re-described, while 98 genuinely held-out robots score 1489.6. A new description can therefore be more damaging than a new robot. Tracing the failure reveals that axis reversal alone reproduces the collapse, joint-angle zero changes are nearly harmless, and reordering lies between them; moreover, changing joint-state and torque coordinates alone is sufficient to cause the failure, while changing description-derived features alone is not. The same phenomenon appears in ModuMorph and an unrelated PyBullet framework. Yet it is not irreversible: exact two-description transport restores the original controller, and training across equivalent axis conventions raises retained return under axis reversal from 3.6% to 80.6%. Together, these results separate mechanism robustness from representation robustness and show that cross-embodiment evaluation should test both.

Tue 6 OctRoboticsMachine Learning
The gist
A robot’s physical parts can be described in different but equivalent ways, like naming joints or setting zero positions differently. The authors found that changing these descriptions alone, without changing the robot itself, can cause robot control programs to fail badly. They show that some description changes, such as flipping joint axes, harm performance more than using new robots altogether. Their experiments suggest robot behavior depends heavily on how the robot is described, not just its physical design. Training across different descriptions can fix the problem and make robot controllers more reliable.
Open → 2610.07597v1

Humanoid robots improve whole-body tasks with self-evolving motion imitation

InterMimicGen: Scaling Humanoid Loco-Manipulation through Self-Evolving Motion Imitation

Abstract: Captured human-object interactions provide rich supervision for humanoid loco-manipulation, but they are sparse, heterogeneous, and not directly executable by robots. We introduce InterMimicGen, a self-evolving motion-imitation framework in which robot motion data and a tracking policy improve each other. First, we consolidate motion-captured human-object interaction datasets and retarget them into humanoid robot references while preserving whole-body coordination and dexterous hand-object relationships. This produces a large and diverse humanoid robot reference collection for dexterous whole-body loco-manipulation. Second, we train a physics-based generalist tracker that executes these references in simulation on a humanoid with dexterous hands, covering a scale and diversity beyond prior humanoid tracking systems for loco-manipulation. Third, we close a data flywheel: each round makes small, task-preserving changes to where an interaction takes place and how the body performs it, fine-tunes the tracker on them, and keeps only the variants whose simulated execution completes the task, which seed the next round. With more iterations, these small edits compound into broader coverage around the sparse original demonstrations while preserving task semantics and motion quality. Experiments show contact-preserving retargeting across robot configurations, broad tracking with a single generalist policy, executable motions that keep growing over augmentation rounds, and transfer to real robots. InterMimicGen provides a unified path from heterogeneous human demonstrations to a continually expanding motion resource for humanoid robot learning.

Mon 5 OctRoboticsComputer Vision and Pattern RecognitionGraphics
The gist
Humanoid robots can learn to perform complex tasks by watching humans, but the real human movements are hard for robots to copy directly. The paper’s authors created a system called InterMimicGen that turns various human motion data into robot-friendly movements and trains a robot to perform them in simulation. This system improves itself by tweaking these movements slightly and keeping only the successful versions, which helps the robot learn more tasks over time. Their work aims to make robots better at coordinating their whole bodies and hands during tasks, and they showed it can transfer to real robots.
Open → 2610.06850v1

Autonomous mobile robot dataset captures logistics jobs and operations

Autonomous mobile robot operations logistics: a dataset of jobs, dispatch events and robot states

Abstract: Autonomous mobile robots (AMRs) increasingly perform material transport in production logistics, where their operation is governed by job generation, dispatching and robot control. We present MoRoOp, a dataset of AMR operations recorded in a laboratory kit preparation and supply scenario over nine eight-hour shifts. During each shift, an AMR executed stochastically generated kit supply, empty-box refill and charging jobs. The dataset links job specifications, the operations constituting each job, dispatch events documenting operation state transitions and outcomes, and robot-state observations comprising position, orientation, velocity, per-wheel state of charge and diagnostics. It contains 1,382 jobs, 4,815 operations, 19,352 dispatch events and 140,386 robot-state observations together with the kit specifications used during job generation. The dataset was recorded in an operating laboratory environment, and technically valid observations of delays, obstructed navigation and unsuccessful operations were retained. Both raw and cleaned robot-state tables are provided. Documented reuse directions include the evaluation of AI agents on operational decision records, disturbance detection, operation prediction, data-driven simulation and event-log analysis.

Thu 1 OctRobotics
The gist
Moving items with robots in factories involves lots of tasks and decisions, like picking jobs and navigating obstacles. The authors collected detailed data from a robot working in a lab for many hours, including what jobs it did, how it got dispatched, and its movement and battery status. Their dataset also records failures and delays, giving a realistic picture of robot work. This data helps others test decision-making systems, detect problems, and simulate robot behavior.
Open → 2610.02428v1

Tactile curiosity improves robot learning for manipulation tasks

Tactile Curiosity Drives Robot Interaction

Abstract: Mastering robot manipulation skills via reinforcement learning (RL) remains largely sample-inefficient. The most common RL algorithms rely on random action sampling to discover new strategies, resulting in agents that allocate most of their training budget to motions in free space, away from the contacts from which manipulation skills emerge. Existing intrinsic motivation methods based on model disagreement or epistemic uncertainty improve on isotropic noise, but they can also reward uncertainty in functionally irrelevant transitions, such as erratic motions in free space. In this work, we argue that tactile feedback provides a natural signal for exploration, and introduce TacEx, a framework that incorporates touch into epistemic uncertainty-driven exploration by decomposing model uncertainty across sensory modalities and directing curiosity toward the tactile channel. By anchoring curiosity to the sense of touch, TacEx drives the robot to discover complex contact dynamics, learning to manipulate and grasp objects without task rewards or expert demonstrations during exploration. The interaction-dense dataset collected through this tactile-driven curiosity supports offline learning of downstream pick-and-place policies without additional environment interaction. We further use tactile-driven exploration to post-train vision-language-action (VLA) models. Although the VLAs are initially pre-trained without tactile feedback, post-training with TacEx substantially improves downstream performance while remaining highly sample-efficient.

Wed 30 SeptRoboticsArtificial Intelligence
The gist
Robots usually learn by trying random actions, but this often wastes time moving without touching anything useful. The authors propose a way to make robots curious about what they feel through touch, guiding them to explore and learn manipulation skills better. By focusing on tactile feedback, the robot discovers how to grasp and move objects without needing help or rewards. This method also helps improve robots that already use vision and language to understand actions.
Open → 2609.40134v1

Large language models enable better robot navigation using standard map images

Spatial and Semantic Reasoning for LLM-Driven Robot Navigation via MCP

Abstract: Large language models (LLMs) are increasingly used as natural-language interfaces for robotic systems, yet their integration with Robot Operating System (ROS)-based navigation remains limited by two gaps. First, navigation data such as occupancy grids are represented as raw geometric messages that are difficult for LLMs to use directly as spatial or semantic context. Second, adding LLM-driven capabilities often requires custom wrappers or robot-specific interfaces, limiting reuse across systems. To address these challenges, we propose a non-invasive framework that connects LLM reasoning with ROS-based navigation through a navigation-oriented representation layer, exposed through the Model Context Protocol (MCP) as standardized, reusable tools so that any MCP-compatible LLM can access them without robot-specific wrappers. The visual map modules transform occupancy grids into metric, pose-aware images for goal reasoning, while the semantic annotation modules record waypoint-level observations with robot poses. We evaluate the framework on three tasks: autonomous mapping, spatial reasoning-based navigation, and semantic reasoning-based navigation. The results show that the evaluated LLM backends use these representations to achieve over 97% map coverage and select spatial or semantic navigation targets from natural-language instructions in a simulated indoor environment. This demonstrates representation-mediated LLM navigation without modifying the existing ROS navigation stack.

Wed 23 SeptRobotics
The gist
Robots use maps to move around, but the raw map data is hard for language models to understand directly. The authors created a new way to convert robot maps into easy-to-read images and notes that language models can use to make navigation decisions. Their method works with usual robot software and doesn't need special robot-specific programming. In tests with a robot simulator, language models guided the robot to cover maps fully and followed natural language instructions accurately.
Open → 2609.27340v1

Uranus builds fast scalable robot simulation with online control

Uranus: Building the Next-Generation Simulation Infrastructure for Embodied AI

Abstract: Scalable simulation is essential for robot data generation, policy training, evaluation, and safe iteration, yet real-world interaction is costly and conventional simulators require labor-intensive construction. We present Uranus, a data-driven robot simulator built around a joint-trajectory-conditioned autoregressive diffusion model. Uranus offers three key capabilities: (1) streaming, open-ended rollout, which receives future joint-position trajectories online and autoregressively generates one latent frame per step, corresponding to four RGB frames, without a fixed horizon; (2) low-latency generation, achieving 24 FPS after inference optimization; and (3) scalable, extensible robot control, providing a unified interface for synchronized multi-view generation across diverse robot embodiments and camera configurations. We conduct comprehensive quantitative and qualitative evaluations on both in-distribution and out-of-distribution data, providing an objective assessment of Uranus and clearly identifying its current limitations. We release the code and model weights to empower the community with practical tools and insights.

Mon 21 SeptRoboticsArtificial Intelligence
The gist
Simulating how robots move and interact with their environment takes a lot of work and time. The authors created Uranus, a new kind of robot simulator that uses a special machine learning model to generate robot movements smoothly and quickly as it receives commands in real time. Uranus can produce realistic video frames at 24 frames per second and supports many different robot types and camera views. The authors tested the system thoroughly and shared the code and model so others can use and improve it.
Open → 2609.24815v1

Taxonomy clarifies robot responses to incomplete human instructions indoors

From Wizard-of-Oz Human-Robot Dialogue Collection to a Taxonomy of Robot Response Decisions: A Retrospective Analysis of Assistive Pilot Interactions

Abstract: Robots that follow natural-language instructions in everyday indoor environments must act on incomplete human utterances. Instructions often omit essential information, such as the identity of an out-of-view object, an intended destination, or the user's goal. Existing datasets contain little real-world situated dialogue and provide few practice-grounded criteria for deciding when a robot should act, confirm, clarify, or refuse. We retrospectively analyze a pilot Wizard-of-Oz study in which five participants performed everyday indoor tasks, including door opening, drawer opening, feeding, drinking, and cleaning, with a wheelchair-mounted mobile manipulator while the wizard responded without a formal communication policy. This preserved authentic user behavior but produced inconsistent robot-side decisions, motivating an explicit decision scheme. From 40 episodes, we derived a hierarchical taxonomy of six response modes (ANSWER, REPORT_DONE, REFUSE, CONFIRM, CLARIFY, ACT) and four ambiguity types (intent, referential, spatial, intelligibility). Two human annotators and an AI annotator applied the scheme to the pilot data. Clean-label rates were 91% and 89%, and Cohen's ranged from 0.72 to 0.95 across decision-point, mode, and ambiguity levels for both human-human and human-AI comparisons. Fine-tuning LLaVA-1.6-7B on taxonomy-derived labels for ACT and CLARIFY indicates the feasibility of training vision-language models using annotations from our taxonomy. Remaining boundary cases in decision-point identification and REPORT_DONE motivate a constrained protocol for more consistent dialogue collection.

Wed 16 SeptRobotics
The gist
Robots helping people indoors often get unclear directions because humans leave out details. The authors studied how a person pretending to be a robot responded during tasks like opening doors or feeding. From this, they created a system to categorize robot replies, like when to act, ask questions, or refuse. They tested how well humans and AI could apply these categories and explored training AI to decide by itself.
Open → 2609.19447v1

Gated residual coordination improves humanoid robot hand and body movement

Gated Residual Body-Hand Coordination for Whole-Body Humanoid Teleoperation

Abstract: Whole-body humanoid teleoperation commonly combines a motion-tracking policy with a separate dexterous-hand retargeter. However, independently generated commands do not explicitly preserve body-hand geometric relations, leading to mismatches in relative wrist poses and fingertip positions during bimanual interaction. We present a gated residual coordination framework that keeps both modules frozen and applies bounded corrections to their outputs. A motion-conditioned action gate allocates correction authority across joint groups, while reference-geometry-dependent reward gates emphasize relevant interaction objectives during training. To establish the nominal body controller on Agile One, we introduce multi-pose morphology calibration that jointly estimates triaxial scales and effector-local offsets, together with staged motion dataset curation for training a SONIC-based tracker. The residual policy uses human motion references, initial commands, and robot proprioception without explicit object or contact observations. In simulation, it reduces wrist and fingertip geometry errors by 39.2-56.3% over direct composition on held-out GRAB motions, while preserving whole-body tracking on AMASS, with success rates of 89.03% without residual coordination and 89.29% with it. Ablations characterize the contributions of reward gating, adaptive correction authority, and separate body and hand correction heads.

Wed 16 SeptRobotics
The gist
Controlling a humanoid robot's whole body and hands at the same time is tricky because the commands for the body and hands often don't line up perfectly. The authors developed a system that applies careful corrections to keep the body and hands working together naturally without changing the main control parts. This helps reduce errors in the robot’s wrist and finger positions, making its movements more coordinated while still following the original motion. They tested their method in simulations and showed it can improve the robot's interaction with objects using just motion and internal sensing, without needing extra information about what it’s touching.
Open → 2609.18763v1

Visual embodiment structure improves robot control and task learning

Rethinking Visual Embodiment Dependence in Visuomotor Policies

Abstract: Visuomotor policies observe both the task scene and the acting embodiment, allowing embodiment-specific visual cues to influence action prediction. We study this phenomenon as visual embodiment dependence (VED) and show, through cue-conflict interventions across representative policies, that visible robot configuration can become a shortcut to task progress. Rather than eliminating VED, we argue that it should be structured around embodiment information that supports control and generalization. We realize this through embodiment canonicalization in 3D point clouds, replacing the original embodiment with a canonical end-effector representation (CER) that preserves control-relevant geometry while abstracting embodiment-specific morphology. Its editable form further enables configuration-decorrelation augmentation for unfamiliar robot configurations. Experiments show that embodiment canonicalization substantially improves human-to-robot policy transfer without robot demonstrations, while simply removing the embodiment is insufficient without preserving control-relevant geometry. We further find that CER itself can become a configuration shortcut when robot configuration becomes decoupled from task progress; configuration-decorrelation augmentation mitigates this failure mode and restores robust recovery without sacrificing performance on seen configurations. Together, these results show that robust visuomotor learning benefits from structuring, rather than removing, visual embodiment information. Project website: https://tonyfang.net/ved

Tue 15 SeptRobotics
The gist
When robots learn tasks by watching scenes and their own parts, they might rely too much on how their body looks instead of focusing on the task. The authors show that instead of removing the robot’s visual body, it’s better to represent it in a standard way that keeps important control details but hides robot-specific shapes. This helps robots learn better from tasks done by humans and handle new robot designs more robustly. They also add training tricks to avoid robots cheating by just looking at their configuration rather than the task progress.
Open → 2609.16815v1

Conditional timed partial orders speed up robot task planning

Conditional Timed Partial Orders: An Expressive and Interpretable Framework for Robot Task Specification and Planning

Abstract: Timed Partial Orders (TPOs), originally proposed for workflows, provide an interpretable framework for robot task specification with planning algorithms based on mixed-integer linear programming (MILP). However, TPOs are limited in expressivity, capturing only partial-order events with simple timing constraints. In this paper, we introduce Conditional TPOs (cTPOs), which extend TPOs with richer relative-timing constraints and conditional event activations based on environmental conditions. We show that planning for cTPOs also reduces to an MILP problem; however, the added expressivity results in significantly larger MILPs that can become computationally intractable. To address this challenge, we propose a decomposition algorithm that partitions a cTPO into smaller sub-TPOs, yielding a sequence of smaller MILP problems. We prove that this decomposition is complete and preserves plan optimality while improving the interpretability of complex tasks. Experimental results demonstrate the effectiveness of cTPOs as a task specification framework and the efficiency of our decomposition approach, achieving up to four orders of magnitude speedup over the monolithic MILP.

Mon 7 SeptRoboticsFormal Languages and Automata Theory
The gist
Robot tasks often need clear instructions about the order and timing of actions. The authors extended an existing way to describe robot tasks, called Timed Partial Orders, by adding conditions and more detailed timing rules. This makes the task descriptions more flexible but also harder to solve computationally. To fix this, they broke down big problems into smaller pieces that are easier to handle, making planning much faster without losing accuracy. Their tests show this approach can speed up planning by thousands of times while keeping tasks understandable.
Open → 2609.07905v1