Papers for

robot control engineers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

World models generate improved experience for offline robot policies

Generating Critic-Guided Synthetic Experience in Latent World Models

Abstract: Learning from offline datasets is limited in scalability, as collecting new experience is often costly or unsafe in the real world. World models offer an alternative way to generate new experience through a learned dynamics model, yet learning policies in latent space is limited due to compounding errors in imagined rollouts. To this end, we propose using the world model as a local transition generator, leveraging critic guidance to choose actions inside that model to generate synthetic experiences. We study action refinement methodologies, introducing Critic-Guided Synthetic Experience (CGSE) generation in world models. Specifically, CGSE augments an existing offline dataset by sampling candidate actions using an offline policy and refining each to maximize a first-order approximation of the $Q$-function within a trust region. We then use a frozen latent world model to imagine the future state given each updated action, and use the critic to choose the best one, appending this synthetic transition to the replay buffer. Our method requires no additional real-environment interaction or test-time planner, as it generates synthetic experience offline using the world model. On three OGBench manipulation tasks, CGSE improves an offline policy by a relative improvement of $7.1\%$ on the average success rate.

Thu 29 OctMachine LearningArtificial Intelligence
The gist
Collecting new data for training robots or AI agents can be expensive or risky in real life. The authors work on using world models, which simulate how things change over time, to create artificial experiences that improve learning. They developed a method where a critic—a model that judges action quality—is used to pick and refine actions inside the simulated world before adding these examples to the training data. This approach boosts performance in robotic manipulation tasks without needing more real-world trials.
Open → 2610.39100v1

Spiking neural networks stabilize periodic motion in underactuated robots

Spiking Neuromorphic Control for Periodic Orbits Stabilization in Underactuated Robots: A Greedy Lyapunov-Based Approach

Abstract: This work investigates the use of event-based spiking controllers, implemented as spiking neural networks, for trajectory tracking in underactuated robots. Specifically, we address the problem of stabilizing a prescribed periodic orbit. It is well known that, by applying the method of transverse linearization to the robot dynamics in a neighborhood of a periodic trajectory, the dynamics can be approximately described by a linear time-varying system with periodic coefficients (a periodic LTV system). The problem of orbital stabilization can therefore be reduced to stabilizing the trivial solution of this LTV system. We show that a previously proposed Lyapunov-based greedy design, originally developed for LTI systems, can also be extended to the corresponding linearized transverse dynamics, which take the form of a periodic LTV system. This enables the construction of a feedback controller for orbital stabilization of the robot. The effectiveness of the proposed method is demonstrated on the trajectory stabilization problem for the butterfly robot.

Tue 27 OctRobotics
The gist
Controlling robots with fewer motors than joints is tricky, especially when trying to keep them moving in a repeating pattern. The researchers study how to use spiking neurons—tiny circuits that mimic brain signals—to guide these robots smoothly along desired paths. By translating the robot’s complex movement into simpler steps, they design a controller that learns moment by moment how to keep the robot steady on its repeating track. They test this approach on a robot called the butterfly robot and show it can reliably follow its path.
Open → 2610.37310v1

Accelerating multi-view 3d geometry with lighter attention mechanisms

GPA-VGGT: Geometry-Preserving Acceleration of VGGT

Abstract: Feed-forward geometry models such as VGGT recover camera parameters, depth maps, and point maps from one to hundreds of views in a single forward pass, making them powerful backbones for multi-view 3D perception. However, VGGT's global attention is quadratic in the number of frames and dominates end-to-end runtime on long sequences, which has motivated a range of training-free efficiency methods such as token merging. These methods, however, come with a drop in accuracy that makes them hard to use in place of VGGT. We propose GPA-VGGT, a training-free acceleration method that replaces global attention in early blocks with frame attention augmented by a shared cross-frame K/V set. In the remaining blocks, we merge only keys and values while preserving all queries, restore the attention mass of the tokens each merged key represents, and randomly project the matching metric to 128 dimensions to reduce matching cost. Across diverse multi-view geometry tasks, GPA-VGGT provides training-free acceleration at speedups comparable to prior work while preserving the information required for precise geometric estimation, retaining over 99% of VGGT's pose accuracy under AUC@3.

Tue 27 OctComputer Vision and Pattern Recognition
The gist
Multi-view 3D perception systems need to process many images to understand a scene’s shape and camera positions. The VGGT method does this in one step but slows down a lot as more images are added due to complex calculations. The authors developed GPA-VGGT, which speeds up VGGT without needing extra training by simplifying these calculations early on while keeping important details. This keeps the 3D estimates almost as accurate while making the process much faster.
Open → 2610.37000v1

Smooth gradients improve legged robot walking control and planning

Differentiable Contact Dynamics for Legged Locomotion: Policy Learning and Contact-Implicit Trajectory Optimization

Abstract: This paper proposes a smooth analytic gradient for frictional contact and applies it to policy learning and contact-implicit trajectory optimization for legged locomotion. The exact contact sensitivity is available in closed form but describes changes within the current contact mode, providing no first-order information toward neighboring contact modes. The proposed gradient replaces this sensitivity only in the backward pass, leaving the hard-contact forward solve unchanged and the realized rollout feasible. A central-path relaxation of the de Saxcé-corrected second-order-cone complementarity problem for Coulomb friction smooths the conditions defining the separating, clamping, and sliding modes. The relaxed sensitivity is evaluated at a point on the central path obtained by a Newton iteration initialized from the hard-contact solution. The sensitivity remains continuous through contact-mode transitions and converges to the exact sensitivity as the relaxation vanishes. Using 100,000 samples and less than 2 min of training on a desktop PC, policy learning with the proposed gradient produces a policy that transfers to hardware without fine-tuning. On hardware, the robot under a trotting policy withstands pushes and traverses debris and slippery ground. In trajectory optimization, the proposed gradient reaches contact-rich solutions missed by exact and Signorini-relaxed gradients. Project page: https://diffcontact.github.io

Mon 26 OctRobotics
The gist
Making robots that walk on legs involves dealing with tricky contacts between feet and ground, which can be hard for computers to plan smoothly. This paper offers a new way to calculate smoother guidance—called gradients—when a robot's foot touches or slips on the ground. The smoother guidance helps robots learn stable walking behaviors faster and plan movements that include different ways of touching the ground. The authors show that using their method, a walking robot can handle tough real-world challenges like slippery surfaces and uneven ground.
Open → 2610.35852v1

Delay randomization improves delay tolerance of humanoid motion controllers

Delay Randomization Widens the Delay Margin of Learned Humanoid Motion Trackers

Abstract: Learned humanoid controllers are usually trained in one simulator and then checked in a second one before they go near hardware. We ask what that check says about command delay. We train motion-tracking policies for a 24-joint humanoid on five motions taken from monocular video and test them in an independent CPU MuJoCo harness, which we first verify against the training model. Every policy completes its motion in at least 26 of 30 episodes under the training randomization. With 40 ms of constant command delay, however, survival drops to 15% on average (0% - 45%), and the delay margin is only 20 - 30 ms. Randomizing the delay during training (0 - 50 ms) lifts survival at 40 ms to 95% on average (75% - 100%) and moves the margin to 50 - 70 ms. A control arm trained for the same number of extra iterations without delay does not improve, so the gain comes from the randomization itself. This holds for all five motions with three training seeds each, and a policy trained in PhysX behaves the same way. The price is slightly looser tracking (0.02 - 0.06 rad on four motions) and, for the most dynamic motion, less robustness to other perturbations. We also show that the first sim-to-sim gap we saw came from the harness, a missing floor and a wrong timestep, and not from the physics engines. The 0 - 20 ms delay ranges common in locomotion training lie inside the margin an unrandomized tracker already has. The benefit comes from randomizing beyond it, so the range should be set from the robot's measured latency, and delay margin belongs next to other sim-to-sim metrics.

Fri 23 OctRobotics
The gist
Humanoid robots that follow learned motions often experience delays when receiving commands, which makes them perform worse or fail. The authors trained controllers by sometimes including random delays during training, which helped these controllers handle longer delays later on without failing. This approach increased the range of delays the controllers could handle, but made their tracking of motions slightly less precise. They also found that some earlier differences between simulators were not due to physics but to simulation setup details.
Open → 2610.32277v1

Vision language action models signal task failure before breakdown

Do VLAs Know They Will Fail? Interpretable Success and Failure Patterns in Physical AI Systems

Abstract: Vision-Language-Action (VLA) models have emerged as powerful general-purpose robot policies, but their successes and failures remain difficult to diagnose and understand. In this work, we show that sparse autoencoder (SAE) representations of OpenVLA activations contain outcome-predictive information and can be used to predict task failure before the robot reaches an unrecoverable state. Moreover, to provide insights into what common behaviors may contribute to these outcomes, we identify individual SAE features whose strong activations are associated with specific stages of success or failure, and uncover recurring behavioral patterns by localizing temporal intervals where these features exhibit significant activation changes. We further develop a VLM-based interpretation pipeline to generate behavioral descriptions of these patterns and validate their correspondence to feature activations on held-out intervals.

Fri 23 OctRoboticsArtificial IntelligenceMachine Learning
The gist
Vision-language-action models are robots that use images and language to decide what to do. The authors show that by looking at internal signals of these robots, it is possible to predict if a robot will fail its task before it gets stuck. They found specific signal patterns linked to success or failure and created ways to explain the robot's behavior during these times. This helps us understand why some robot attempts fail and could improve robot reliability.
Open → 2610.32208v1

Embodied foundation models unified to improve robot actions

Unifying Embodied Foundation Models via a Model-to-Policy Formulation

Abstract: Vision-language-action (VLA) models and world-action models (WAMs) have emerged as promising approaches to robot learning and control. These models integrate multimodal perception and action generation, but differ in their architectures, output representations, and learning objectives. In this paper, we present a unified, architecture-agnostic probabilistic formulation for embodied foundation models based on stochastic kernels. Our formulation has two key components: (i) we characterize embodied foundation models through conditional output distributions and action-realization kernels, which together induce task-conditioned robot policies; and (ii) we connect pretraining and post-training to policy deployment, accommodating model-specific pretraining objectives and offline or online post-training strategies. We instantiate the formulation for OpenVLA and DreamZero, covering autoregressive action-token prediction and joint video-action generation.

Fri 16 OctRoboticsArtificial IntelligenceMachine Learning
The gist
Robots need to understand what they see and decide how to act. Different robot models do this in various ways, making it hard to connect or improve them together. The authors propose a new way to describe and unify these robot models so they all work from a common idea. This helps combine learning methods and better adapt robots to new tasks. They tested their idea on two existing robot models and showed how it applies to different kinds of robot action planning.
Open → 2610.23053v1

Robust guarding of polyomino shapes with movable guards

Restless Guards: Robust Guarding of Polyominoes

Abstract: The art gallery problem is a fundamental problem whose practical applications have inspired numerous variants. In this work, we investigate a setting in which full coverage of the gallery shall be ensured despite locational uncertainty of the guards. We study this problem for polyominoes under rectangular visibility, where a $k$-restless guard assigned to a cell may move to any cell within distance at most $k$. While all polyominoes are guardable for $k\in\{0, 1\}$, the same is not true for all $k\geq 2$. However, for a given polyomino $P$ and integer $k$, we present an algorithm that identifies a $k$-restless guard set of $P$ in polynomial time, or decides that no such set exists. For the cases $k\in\{0,1\}$, where $k$-restless guard sets always exist, we provide tight worst-case bounds on the number of $k$-restless guards that are always sufficient and sometimes necessary to guard a hole-free polyomino with $n$ vertices or $m$ cells. Moreover, we show that computing a minimum guard set is NP-hard. Complementing the hardness, we provide polynomial-time algorithms for solving the problem optimally for histograms, and leverage this to obtain a $20k$-approximation for polyominoes without holes.

Thu 15 OctComputational GeometryData Structures and Algorithms
The gist
The paper looks at how to place security guards on a grid-like shape to watch the whole area even when guards can move a little bit from their spot. The authors study this for shapes made of square cells and where guards see in straight lines through rows or columns. They find that allowing small movements helps in covering shapes fully for certain movement limits but not always for larger ones. They also provide methods to find the minimum number of guards needed and show when this is hard to compute. For simpler shapes without holes, they give efficient solutions and approximations.
Open → 2610.21793v1

Multimodal control improves robotic arm therapy engagement and effort

A Flow-Theoretic Multimodal Control Framework Fusing Mental and Physical Engagement for Upper-Limb Rehabilitation

Abstract: While Assist-as-Needed (AAN) control aims to maximize user participation in robot-assisted training, existing methods relying solely on physical metrics cannot distinguish muscular inability from mental disengagement, leading to suboptimal interventions. To address this ambiguity, a Flow-theoretic multimodal AAN framework is proposed, fusing EEG-based mental engagement and sEMG-based physical engagement into an adaptive impedance control strategy. A four-quadrant fuzzy decision engine translates these dual physiological metrics into continuous control policies. This enables the upper-limb exoskeleton to inject feedforward assistance during intent-execution mismatch (e.g., motor output attenuation despite mental focus) and escalate guiding stiffness to penalize mental distraction. The framework was validated with healthy participants against a state-of-practice sEMG-only AAN baseline. Kinematically and mechanically, the proposed framework reduced robotic energy expenditure by 71.9\% while maintaining precise trajectory tracking, confirming it elicited greater voluntary human effort. Physiologically, the system increased the time spent in the neuroplasticity zone (Flow State Ratio) from 4.6% to 42.8%. These results validate that decoupling and jointly regulating mental and physical states enhances the selectivity of robotic assistance, mitigating user passivity and sustaining the engagement crucial for motor learning.

Thu 15 OctRobotics
The gist
Robot-assisted arm therapy often struggles to tell if someone is not moving because their muscles are weak or if they are just not paying attention. The authors introduced a new method that combines brain signals showing mental focus with muscle signals to better guide the robotic arm. This system adjusts the help it provides based on both mental and physical effort, encouraging patients to stay engaged and work harder. Tests showed it reduced energy use by the robot and kept people in a focused state much longer, which is important for recovery.
Open → 2610.21590v1

Robot learns to walk better after leg damage without resetting

Persistent Profile-Guided Reset-Free Recovery Improves Final Gait Selection after Irreversible Leg Damage

Abstract: Damage-recovery algorithms often rank gaits from short trials, yet a gait that performs well briefly may not remain effective during continued execution. In reset-free recovery, each trial also changes the physical state inherited by the next. We introduce persistent profile-guided reset-free recovery (PGRR), which predicts multiple outcomes over repeated gait execution and uses conservative uncertainty-aware estimates to guide short online probes without resetting the robot. Across 108 matched MuJoCo tasks spanning leg amputation, distal fracture, and joint lock, PGRR improves final quality by 0.106 and slot-averaged recovery quality (SRQ) by 0.044 relative to an IT&E-inspired Gaussian-process baseline, while reducing low-quality failures by 27.8 percentage points. Final-quality gains occur in all three damage classes. Against no adaptation, PGRR also improves final quality and reduces low-quality failures, whereas the aggregate SRQ difference is inconclusive. Development ablations identify conservative uncertainty handling as the clearest supported contributor to the observed gains. These results show that PGRR can identify stronger post-damage gaits under continuous physical state and that final-gait quality and recovery-summary performance provide complementary evidence for evaluating reset-free recovery.

Wed 14 OctRobotics
The gist
Robots sometimes get damaged legs and need to find new ways to walk. The authors created a method called PGRR that helps robots try different walking styles continuously without restarting from the same spot each time. This method uses careful predictions to pick better movements and avoids choosing gaits that seem good only for a short time. Testing in many simulated damaged leg scenarios, PGRR found stronger walking styles more reliably than previous methods. This helps robots adapt better to serious leg damage while moving.
Open → 2610.20838v1

Interpolation reduces abrupt robot motion caused by delayed commands

JitterVLA: A Replay Benchmark for Motion Quality in VLA-Driven Human-Robot Manipulation

Abstract: Vision-language-action (VLA) policies issue sparse targets whose arrival at the robot controller is delayed and irregular, so a reasonable predicted action can still produce abrupt motion near people. Remedies are evaluated on different streams, delays and metrics. JitterVLA replays identical commands and arrival traces through alternative controllers, isolating the controller's effect. We apply it to synthetic reaches, hand trajectories from 20 people, three open VLA pipelines (838 episodes) and a physical UR3. Interpolation lowered modeled p99 jerk for every source, by 83% (615 to 106 m/s^3) on human motion with 200 ms delay, at a 2.0 cm tracking cost, and the effect replicated on 598 new episodes; jerk-limited generation lowered jerk further at a larger cost. Entropy-discontinuity associations had opposite, replicated signs in two pipelines, and entropy gating helped in one but not the other. On the UR3, interpolation lowered accelerometer-derived jerk by 3.0-14.0% in all six exploratory pairs, while a position-derived metric did not resolve the contrast. JitterVLA supports screening before human-centred studies; code and data will be released.

Fri 9 OctRoboticsHuman-Computer InteractionMachine Learning
The gist
Robots controlled by vision and language sometimes move abruptly because the commands they receive come with delays and irregular timing. The authors created a way called JitterVLA to replay the same commands through different controllers to see how each controller affects motion smoothness. They found that interpolating commands can greatly reduce jerky movements even when there are delays, making robot motion safer near people. Their method works both in simulated reaches and in real robot arms, helping screen robot controllers before testing with humans.
Open → 2610.13628v1

Safe reinforcement learning improves robot control with less risk

FAITH: Feasibility-Aware Safety-Filtered RL for High-Dimensional Systems

Abstract: Safe reinforcement learning commonly places safety and task performance in the same policy objective, where they can introduce competing updates. Safety filters separate them at action execution, but classical designs require an analytic safety function and dynamics model, and standard minimal-intervention filters are myopic to long-horizon task return because they minimize only instantaneous action deviation. Hard projections are also undefined when no safe action exists. We present FAITH, a feasibility-aware, model-free framework that approximates the optimal state-action safety value and amortizes minimal-intervention filtering with a feedforward network. The task policy optimizes the task return through the filtered dynamics, which recovers the feasible constrained problem without a competing safety term in the task-policy update. When no action satisfies the learned safety condition, the same filter approaches the action with minimum predicted peak harm. On a double integrator example and a Safety Gym environment, FAITH achieves the highest return among methods with no feasible-start violations and matches the lowest harm from infeasible starts. On a 29-DoF humanoid, it reaches a 99.95% safety rate while retaining 97% of the unfiltered return in Walking-Avoid, and obtains the highest measured safety rate in Push-Avoid by learning to sacrifice balancing and fall away from the protected region. The same policies are also demonstrated on a real-world Unitree G1 humanoid.

Thu 8 OctRoboticsMachine Learning
The gist
Safe robot behavior is hard because safety and task goals can conflict during training. The authors designed FAITH, a method that separates safety checks from task learning and predicts the safest actions without needing a detailed model of the robot's dynamics. When no fully safe action exists, it picks the one with the least risk. They tested this on simulated robots and a real robot, showing it keeps robots safe while performing tasks well.
Open → 2610.12432v1

Behavioral models improve task detection for better robot control

Task Inference Beyond Least Squares in Behavioral Foundation Models

Abstract: Behavioral Foundation Models (BFMs) aim to solve a wide range of downstream tasks without test-time policy learning by inferring a task vector from the reward function. While efficient, the retrieved policies are often suboptimal because of how this task vector is inferred, typically with ordinary least squares (OLS). OLS minimizes reward reconstruction error but leaves the ordering of rewards unconstrained, which can bias the successor measure of the retrieved zero-shot policy away from that of the optimal policy. In this work, we propose BLS, an efficient test-time inference method that balances minimizing reward reconstruction error with reducing successor-measure mismatch. Theoretically, we provide a suboptimality gap upper bound characterized by both successor-measure and reward-function residuals. Empirically, we evaluate BLS on top of state-of-the-art BFMs across benchmarks for locomotion, manipulation, and humanoid control. BLS outperforms existing task inference baselines with negligible computational overhead. Project page: https://embodiedai-ntu.github.io/BLS

Sun 4 OctMachine Learning
The gist
Robots often need to figure out what task they should do by understanding rewards or goals, but current methods can guess wrong and lead to less effective actions. The authors introduced a new way called BLS that balances understanding rewards while making sure the robot's behavior matches what truly works best. This new method helps robots perform better on tasks like walking, moving objects, or human-like motions, and it does so without needing extra computing power. They tested it on several challenging robot control tasks and saw clear improvements.
Open → 2610.05350v1

LLM agents improve skills by analyzing their own thought patterns

Rep2Skill: Representation-Guided Skill Self-Evolution for LLM Agents

Abstract: Textual skills enable large language model (LLM) based agents to accumulate reusable procedural knowledge without updating model parameters. Yet existing skill evolution remains largely confined to the text space: an optimizer must diagnose success and failure patterns, and revise skills solely from long execution trajectories and sparse task outcomes. This text-only paradigm leaves the agent's internal representations, which contain rich records of its evolving execution state, outside the skill optimization loop. We ask whether an agent can improve its external textual skills by reflecting on its own internal representations. We introduce Rep2Skill, a representation-guided framework for self-evolution on agent skills. Specifically, upon the collected agent rollouts, Rep2Skill models their internal model representation trajectories to localize turns that deviate from successful execution dynamics, and it further interprets these signals alongside the execution contexts as actionable textual feedback for targeted skill revision. Experiments on two agent environments with two open-source LLMs show that Rep2Skill consistently outperforms text-only approaches in the self-evolution setting, where the same LLM serves as both executor and optimizer without a stronger external model. This establishes a promising direction moving agent self-improvement beyond text-only reflection.

Wed 30 SeptArtificial Intelligence
The gist
Large language model (LLM) agents learn reusable skills to complete tasks without changing their underlying code. However, previous methods improved these skills only by looking at the final text outputs and task results, missing the rich internal thought processes the agent went through. The authors introduce Rep2Skill, a way for agents to look inside their own internal decision-making steps to spot where things went wrong and fix their skills with targeted feedback. Tests show this method helps agents get better faster than just using text feedback alone. This work moves AI agents closer to improving themselves more like humans do, by reflecting on their own thinking.
Open → 2609.39149v1

Multirotors maintain control authority during contact with walls

Residual Wrench Certification and Margin-Aware Control Synthesis for Aerial Physical Interaction

Abstract: We develop a task-relative framework for certifying residual wrench authority after hover and contact loading in multirotors with bounded actuators. Using convex geometry, we derive signed margins for prescribed convex reserves, including Euclidean balls and weighted ellipsoids. We obtain computable reserve certificates from actuator-interiority bounds to support slack-maximizing allocation. To preserve the required reserve, we propose command projection onto a tightened feasible set. We then connect the available reserve to structured gain synthesis and certify a local tracking region that respects actuator limits. Within this region, we establish nominal exponential convergence and robust ultimate boundedness. For the prescribed task and morphology family, we show that the optimized octarotor attains a larger margin than the optimized hexarotor at equal total thrust. We evaluate the proposed framework in closed-loop simulations of sustained rigid-wall contact using a fixed-geometry octarotor and a variable-tilt quadrotor with matched installed thrust. The quadrotor's local certificate admits a higher normalized push limit for the prescribed task family. In tests beyond the realizability boundaries, we observe the predicted loss of authority through rotor-thrust limits for the octarotor and tilt-servo limits for the quadrotor.

Wed 30 SeptRobotics
The gist
Flying drones that gently touch walls while hovering can lose control when their motors hit limits. The authors found a way to check how much control authority remains after the drone pushes against a wall. They use math to ensure the drone stays within safe control commands and can adjust the drone’s movements accordingly. They tested these ideas in simulations comparing drone designs and showed how their method predicts when control is lost.
Open → 2609.38707v1

Tactile sensing improves robot learning for precise manipulation tasks

TaRL: Learning General and Physical Rewards from Tactile Demonstrations

Abstract: Contact-rich manipulation requires robots to sequence precise contacts, maintain stable grasps, and apply directed forces. Reinforcement learning (RL) can acquire such behaviors automatically, but its performance hinges on reward design: sparse rewards reduce the learning efficiency, while dense rewards are hard to specify. Visual reward learning addresses this by inferring rewards from action-free demonstrations. Because it conditions only on visual observations, it fails to capture rewards beyond visual goals. We propose Tactile Reward Learning (TaRL), a framework that learns rewards from tactile demonstrations. TaRL takes a sequence of tactile deformation maps as input, and regresses task-completion progress from both successful and failed demonstrations. Because TaRL captures local robot-object interaction, it provides informative feedback to learn firm grasps and correctly directed forces; meanwhile, it is robust to changes in scene layout such as object position. We evaluate TaRL on four manipulation tasks in simulation and two in the real world. Used as a shaping reward, it substantially improves both sample efficiency and final success rate, raising success on Nut threading from 34% to 56% in simulation and on cube pickup from 37% to 97% in the real world. Combining tactile with visual rewards improves performance further. TaRL also generalizes across object instances: trained on box placement and directly deployed to can placement, it significantly improves policy learning on the new task. Project page is available at https://embodiedai-ntu.github.io/tarl.

Tue 29 SeptRobotics
The gist
Robots often struggle to learn tasks that involve careful contact and gripping because it's hard to design clear goals for them to follow. The authors developed a method that teaches robots by learning rewards from touch-based demonstrations instead of just visuals. This approach helps robots understand how well they are gripping or applying force, improving learning speed and success. Tests show that combining touch and vision helps robots perform better on tasks like threading nuts and picking up objects.
Open → 2609.36785v1

RoboCompiler simplifies consistent robot modeling and control

RoboCompiler: Graph-Native Compilation of Closed-Chain Robots for Consistent Modeling, Control, and Simulation

Abstract: Robots with kinematic loops, coupled actuators, and changing contacts require consistent models of configuration, motion, force, and dynamics. Yet these interfaces are often reconstructed separately for control and simulation, making closure and actuation consistency difficult to maintain. This paper presents RoboCompiler, a graph-native framework that compiles a canonical mechanism graph into a shared mechanical interface. From bodies, joints, frames, inertias, and actuator ports, it constructs closure paths and analytic residual Jacobians, then assembles feasible configurations through rank-checked continuation and correction. A tangent lift maps independent velocities to full robot and task motion, while paired actuator-port maps preserve virtual work. A constraint-curvature correction extends the reduction to accelerations and projected rigid-body dynamics, including floating-base and support modes. Cycle-local evaluation, generated Jacobians, and dependency-aware reuse enable localized updates when closure inputs change. We evaluate physical loops and task-induced constraints on a Komatsu excavator, Unitree Go2, Franka Panda, Kangaroo, and a six-UPS Stewart platform. High-precision constrained-dynamics and independent Pinocchio checks confirm mechanical consistency; MuJoCo and Isaac Sim/PhysX executions demonstrate task performance and model reuse under native contact. For Kangaroo, compilation reduces residual-and-Jacobian evaluation time by 96.7% and closed-loop rollout wall time by 66.8%, with dynamics and control held fixed.

Mon 28 SeptRobotics
The gist
Robots with complex moving parts, like loops and linked joints, need precise models that work the same for controlling and simulating them. The authors created RoboCompiler, which turns a robot's mechanical setup into a shared model that keeps everything consistent across different uses. This approach speeds up calculations and keeps the robot's motions and forces accurate in simulations and real controls. The system was tested on real and simulated robots, showing it works well and reduces computational time significantly.
Open → 2609.35717v1

Humanoid robots learn whole-body tasks from human video and motion data

WB-WAM: Heterogeneous Body-Hand Pre-training for Humanoid Loco-Manipulation

Abstract: Humanoid loco-manipulation demands coordinated body and hand behavior, while conventional robot pre-training data provide limited coverage of such whole-body motion. We present WB-WAM, a World Action Model that incorporates explicit whole-body action supervision into generative video pre-training. A shared physical action space integrates body, root, and dexterous hand annotations from heterogeneous sources, enabling joint video and action learning from 1880.2 hours of partially annotated video and motion data. The resulting priors are refined through PICO mid-training and adapted to robot tasks with auxiliary forward kinematics supervision. We construct WB-Datasets to support these stages with retargeted egocentric human demonstrations and robot trajectories, allowing task-aligned human motion to supplement limited robot data. Evaluations in simulation demonstrate strong whole-body task performance with 81.9% in HumanoidArena, while real-world experiments further validate WB-WAM with 84.0% mean success across five tasks. Moreover, task-aligned PICO mid-training improves downstream task performance while reducing the need for real-robot demonstrations. These results support heterogeneous whole-body pre-training and human motion transfer as a practical route to data-efficient humanoid loco-manipulation.

Mon 28 SeptRobotics
The gist
Humanoid robots need to coordinate their whole bodies and hands to perform tasks, but existing training data often lack this comprehensive motion information. The authors developed WB-WAM, a model that learns from a large amount of human video and motion capture data which includes detailed body and hand actions. By combining diverse sources of human movement data and refining the model with special training steps, they improved robot performance in simulated and real-world tasks. This approach helps robots learn complex coordinated movements more efficiently by using human demonstrations to fill in gaps where robot data is limited.
Open → 2609.34199v1

New method improves decision making in changing bandit problems

Future Information-Directed Sampling for Bayesian Nonstationary Bandits

Abstract: Exploration--exploitation is a central trade-off in bandit learning. While classical algorithms such as upper confidence bound methods and Thompson Sampling effectively balance this trade-off in stationary environments, their exploration strategies mainly reduce uncertainty about the current optimal arm, which can be insufficient in nonstationary settings where future optimal arms may differ substantially from current ones. In this paper, we propose Future Information-Directed Sampling (FIDS), a new algorithm for Bayesian nonstationary bandits that explicitly explores to gather information about future optimal arms. We show that FIDS achieves regret comparable to Thompson Sampling up to a small constant factor, while being able to exploit predictive information structures that conventional exploration objectives fail to capture. To address the practical difficulty of posterior inference, we further propose a supervised-learning-based approximation framework that learns the FIDS policy from offline data, and demonstrate its effectiveness on synthetic benchmarks.

Sun 27 SeptMachine Learning
The gist
When making choices over time, like picking the best option among many, traditional methods look only at what is best right now. This paper presents a new method that tries to learn about what might be the best choice in the future, especially when things change over time. The authors show their method works almost as well as popular ones that assume stable conditions but can better handle changing situations. They also offer a way to teach this method from pre-collected data, making it practical to use.
Open → 2609.33981v1

In-context guidance improves multitask optimization with few trials

In-Context Guidance: Learning Inter-Task Synergies via Numerical Foundational Models for Few-Shot Multitask Optimization

Abstract: Multi-task optimization (MTO) addresses a set of optimization tasks simultaneously, often suffering from inaccurate inter-task relationship estimation under limited evaluation budgets, leading to negative transfer. This paper introduces In-Context Guidance Multitask Optimization (ICG-MTO), a novel framework that leverages numerical foundational models to improve inter-task coupling estimation in few-shot scenarios. Unlike conventional methods that rely solely on scarce observed data, ICG-MTO employs a frozen foundational model to infer auxiliary guidance through in-context learning. The framework operates through three stages: constructing an algorithm-specific in-context query from evaluated solutions, using the foundational model to infer a guidance signal characterizing predictive relationships among tasks, and translating this signal into algorithm-specific guidance for maximum-a-posteriori coupling estimation. This approach provides regularization during the early, data-scarce stages of optimization and gradually relinquishes control as task-specific observations accumulate. We instantiate the framework in multitask Bayesian optimization as ICG-MTBO, using directional fitness-class queries to guide inter-task coupling estimation, and further instantiate it in MFEA-II using decision-space-overlap queries to guide random mating probability estimation. Experiments across synthetic benchmarks and a real-world robot arm control problem, together with evaluations under different acquisition functions and evolutionary multitasking, demonstrate the effectiveness and generality of ICG-MTO for few-shot multitask optimization.

Tue 22 SeptMachine LearningArtificial IntelligenceNeural and Evolutionary Computing
The gist
Optimizing many related tasks at once often struggles when only a few test results are available, causing mistakes from assuming tasks are too connected or not enough. The authors offer a new way called In-Context Guidance Multitask Optimization (ICG-MTO) that uses a powerful pre-learned model to guess how tasks relate to each other without needing lots of examples. This helps guide the optimization better at the start, reducing errors, and gradually lets the real task data take over as more results come in. They tested their method on computer simulations and a robot control problem, showing it works well and can be applied in different optimization algorithms.
Open → 2609.25836v1

Social navigation adapts online with performance preserved by diffusion steering

Performance-Preserving Online Adaptation in Social Navigation via Diffusion Steering

Abstract: In social navigation, modeling the complex interactions between humans and robots is difficult, and deep reinforcement learning has therefore been actively studied. However, because simulation alone cannot fully reproduce diverse scenarios, robot dynamics, and the social conventions that vary across deployment environments, fine-tuning in the deployment environment is promising. In doing so, learning that preserves the base model's performance is required, so as not to compromise the primary objective of navigation, namely avoiding pedestrians and reaching the destination. In this study, we propose a method that applies diffusion steering via reinforcement learning (DSRL), which trains only the noise policy while keeping the diffusion policy fixed, thereby achieving learning that preserves performance. Furthermore, we integrate diffusion-based RL policies trained with multiple seeds to construct the base policy, improving learning performance. Our evaluation shows that, compared with other methods, the proposed method enables efficient learning while preserving performance, and we confirm flexible behavior control through adaptation to social conventions, as well as its effectiveness on a physical robot through hardware-in-the-loop simulation.

Mon 21 SeptRobotics
The gist
Navigating around people is tricky for robots because social behaviors and robot movements vary a lot and are hard to simulate perfectly. The authors propose a way for robots to learn and adapt to new social environments without losing their original ability to avoid people and get to their destination. Their method uses a special kind of learning called diffusion steering where only a small part of the robot’s behavior is updated, keeping the core skill intact. Tests show this allows robots to learn safely and flexibly, even on a real robot connected to simulations.
Open → 2609.24317v1

Compute-efficient branching improves reinforcement learning for complex tasks

EPIG-Tree: Compute-Optimal Branching for Gradient-Efficient Reinforcement Learning

Abstract: Reward-based reinforcement learning for language models, exemplified by Group Relative Policy Optimization (GRPO), collapses an entire stochastic trajectory into a single scalar reward. This is clean and scalable, but it explores and allocates reward inefficiently: a trajectory may contain many causal decisions, recovery attempts, and environment-randomness events, yet every token or action inherits one trajectory-level advantage. We study tree-based rollout construction as a compute-allocation problem for policy-gradient estimation. Our central claim is that branches should be placed not where the policy is merely uncertain, but where an additional branch most reduces uncertainty about the policy gradient per unit of compute. From a law-of-total-variance decomposition of the local policy-gradient random variable, we derive two allocation laws: new branches reduce decision uncertainty, while repeated suffix rollouts reduce continuation uncertainty. The resulting EPIG-Tree score allocates branches using the already computed rollouts. It estimates occupancy- and score-weighted value uncertainty, along with a suffix law $n_e \propto w_e \|\nabla_θ\log π(a_e|h_e)\| σ_e / \sqrt{c_e}$. Empirically, EPIG reduces gradient MSE in cloned-state control, winning in all nine dense continuous-control environments of a 13-environment sweep and recovering the reference gradient direction near-perfectly, and it improves frozen-LLM gradient calibration relative to entropy branching. In online single-turn math, tree-local credit beats flat GRPO, while branch placement is secondary to token-level credit assignment. In online multi-turn Wordle, EPIG attains the highest final win rate (0.850), overtaking flat GRPO, which saturates early at 0.790, and entropy branching as training proceeds, confirming that the gradient-estimation advantage transfers to a stateful, large-action setting.

Thu 17 SeptMachine LearningArtificial Intelligence
The gist
Reinforcement learning methods often treat a whole sequence of decisions as one reward, which can be inefficient. The authors designed a new method called EPIG-Tree that decides where to explore more carefully during learning, focusing effort where it most reduces uncertainty. This approach makes learning more accurate and effective for tasks involving many steps, such as controlling robots or playing games like Wordle. Their experiments show that EPIG-Tree outperforms previous methods in various environments and improves the quality of learning signals in language models.
Open → 2609.20004v1

Neural method improves safety control for high dimensional robots

Gradient-Free Neural Hamilton-Jacobi Reachability for Scalable Safety-Critical Control

Abstract: Hamilton-Jacobi (HJ) reachability provides a principled framework for synthesizing safety certificates and robust controllers for safety-critical robotic systems. However, applying reachability analysis to high-dimensional nonlinear systems remains challenging: classical grid-based solvers suffer from the curse of dimensionality, continuous-time neural solvers require accurate spatial value gradients, and reinforcement-learning-based approaches often suffer from weak boundary anchoring and non-stationary adversarial policy optimization. We propose a discrete-time neural reachability framework for control-disturbance-affine systems that learns backward reachable tubes (BRTs) and backward reach-avoid tubes (BRATs) through Bellman-Isaacs value propagation. Our key idea is to combine equation-driven self-supervision with structured policy learning: rather than computing explicit PDE-gradients, we exploit the bang-bang structure of optimal safety interventions to construct approximate teacher actions from gradient-free value probes, converting adversarial actor learning into supervised policy learning. To stabilize long-horizon value propagation, we leverage the learned actor to train the value function backward from the terminal boundary using a windowed temporal curriculum, where each window is used as the boundary condition for the next window. Across benchmark problems up to 80 dimensions, our method learns accurate reachability value functions while improving stability over existing learning-based solvers. We further demonstrate observation-space scalability on F1-tenth racing with over 16,000-dimensional egocentric inputs. The learned safety filter generalizes zero-shot to unseen tracks and transfers to a physical RC car, achieving real-time robust collision avoidance.

Sat 12 SeptRobotics
The gist
Controlling robots safely, especially those with many parts or sensors, is very hard with traditional math tools because they get too complex. The authors created a new way that uses neural networks to predict safe actions for robots without needing complicated calculations of spatial information. They teach the system by simulating how safety decisions should work, making learning more stable, even for very complex robots with thousands of inputs. Their method worked well in computer tests and controlled a real small racing car to avoid collisions.
Open → 2609.14087v1

Dynin Robotics uses shared model to improve robot action and goal prediction

Dynin-Robotics: Omnimodal Unified Diffusion Vision-Language-Action Model

Abstract: Visual goal and dynamics prediction can provide language-conditioned robot policies with both a target outcome and a representation of action-dependent scene changes. We bring these predictions into action generation and selection through a shared trajectory model. Dynin-Robotics implements this formulation on Dynin-Omni, an omnimodal masked-diffusion backbone, representing language, visual observations, goals, and actions as discrete tokens. By varying conditioning and target spans, the same model learns action prediction, action-conditioned next-observation prediction, terminal goal-state prediction, and trajectory-to-instruction reconstruction. These interfaces support test-time scaling through goal prediction, action-candidate evaluation, and joint refinement of action and future-state predictions. We continually pretrain the model on approximately 1.33 million trajectories from 48 Open X-Embodiment datasets and adapt it separately to downstream domains. On two VLABench tasks, robot pretraining improves adaptation within a fixed Stage-2 step budget, and the full objective mixture improves shifted-instruction success over Policy-only post-training under the same coupled decoder. Combining goal guidance with joint action-next-state denoising further improves shifted-instruction success over action-only decoding; the benefit depends on how the predictions are composed. Dynin-Robotics achieves competitive performance on LIBERO and zero-shot LIBERO-Plus, together with a 78.4% average success rate across four manipulation conditions on a Franka Research 3 robot. An optimized block-parallel implementation accelerates model-side action decoding by up to 29.2x relative to the base implementation under the reported profiling setup. These results support shared trajectory modeling as a common interface for learning complementary robot objectives and composing their predictions during control.

Fri 11 SeptRoboticsArtificial IntelligenceMachine Learning
The gist
Robots need to understand both what to do and how their actions change the world to succeed at tasks described by language. The authors developed Dynin-Robotics, a single model that learns to predict robot actions, future scenes, goals, and instructions all together. It does this by turning everything into tokens and training on a large variety of robot data. This unified approach helps robots perform better in new tasks and speeds up decision-making. The model was tested successfully on several robot benchmarks and a real robot arm.
Open → 2609.13053v1