Papers for

robotic automation teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Action aware model improves long task progress tracking in robotics

Predictive Long-Horizon Progress Modeling with Action-Aware Quantification

Abstract: Quantitative and dense task progress feedback is essential for robotic manipulation across data curation, policy evaluation, and reinforcement learning. Existing progress annotations often rely on sparse keyframe markings derived from subjective manual judgments, limiting their scalability and cross-task consistency, particularly in long-horizon manipulation. To overcome these challenges, we propose action-aware quantification, a unified progress specification that decomposes long-horizon trajectories into object-level subtasks via a vision-language model (VLM) and assigns weighted action credits to capture fine-grained execution dynamics. Using VLM auto-labeling and human calibration, we construct an 800+ hour training video dataset and LOOM, a long-horizon progress benchmark containing 19,618 human-calibrated evaluation frames. To model these dense progress dynamics, we introduce NimCritic, a predictive progress critic trained via Joint Video-Progress Learning. We further incorporate Historical Progress Perturbation to mitigate sequential drift, and design a contrastive Video-Progress Alignment objective that bridges asymmetric expert streams by mutually grounding intermediate visual dynamics and denoised progress trajectories. NimCritic matches or outperforms state-of-the-art process reward models (PRMs) on the established Video Progress Benchmark (VPB) and LOOM, surpassing GVL (GPT-6-Sol) on VPB by about 65% in terminal success accuracy (TSA). On the large-scale RoboArena policy evaluation benchmark, it also retrieves human-selected successful rollouts with about 97% overall recall. These results establish action-aware quantification as a scalable foundation for dense progress feedback in robotic manipulation.

Fri 23 OctRobotics
The gist
Tracking progress in long and complex robot tasks is hard because current methods mark only a few key moments. The authors present an approach that breaks down tasks into smaller subtasks using a vision-language model and assigns detailed scores based on robot actions. They built a large video dataset with human feedback and a new evaluation benchmark. Their method, NimCritic, predicts detailed progress more accurately than previous models and helps find successful robot task executions. This work makes progress tracking in robot manipulation more detailed, scalable, and consistent.
Open → 2610.32776v1

Bimanual robot control improves by aligning vision and physics knowledge

GRASP: Global-Local Representation Alignment and Spatial-Physical Grounding for Bimanual Robotic Manipulation

Abstract: Vision-Language-Action (VLA) models that directly regress joint-space actions avoid the deployment-time cost of inverse kinematics. However, their training objectives remain misaligned with task success, as joint-space losses ignore the unequal physical consequences of joint errors and visual conditioning lacks explicit focus on hand-object interaction regions. To address these issues, we propose a training-phase prior injection framework for fine-grained bimanual manipulation. First, a physics-grounded flow-matching loss maps the implicit clean-action estimate at every noise level through differentiable forward kinematics (FK) and supervises the resulting Tool Center Point (TCP) pose and inter-arm relative pose, with a bounded noise-level weighting that compensates gradient decay. Second, we introduce a global prediction query group distilled from a frozen DINOv3 teacher through future-frame feature prediction, together with a local interaction query supervised by Grounding DINO at the attention level. Both teachers are removed at inference, and the deployment pipeline is that of a standard joint-space flow policy, adding only the learned query pathway. Simulation and real-robot experiments on long-horizon bimanual tasks show that the two priors are complementary across different deployment settings. The full model excels in in-distribution deployment, while the distilled query branch is preferable under strong visual randomization.

Fri 16 OctRobotics
The gist
Controlling robots with two hands is tricky because small mistakes in joint movements cause big problems when interacting with objects. The paper’s authors created a method that teaches robots better by focusing on how their hands interact with objects and considering their physical movement limits. They use smart training guides from other AI models to help the robot understand global and local aspects of the task. This makes robots better at long and complex tasks both in simulations and in real life.
Open → 2610.23123v1

Adaptive verification improves quality of robotic actions in manipulation

VeriVLA: Adaptive Test-Time Verification for Flow-Based VLAs with Latent World Models

Abstract: Flow-based Vision-Language-Action (VLA) policies are inherently stochastic: different noise seeds can produce action chunks of varying quality from the same observation. At test time, however, the policy typically commits to a single sample without an explicit estimate of its quality. In fine-grained manipulation, executing a suboptimal action can irreversibly disrupt task progress and lead to failure. To exploit policy stochasticity, we introduce a training-free two-axis trigger that uses two initial samples to measure within-chunk temporal variation and cross-sample variance, detecting the critical states. Once triggered, a JEPA-based latent world model ranks candidate action chunks by predicting action-conditioned future latent states and the corresponding changes in task progress. By operating in latent space, the verifier avoids pixel reconstruction and enables efficient and more accurate ranking. Across three real-world tasks, $\textit{i.e.,}$ pen insertion, pipetting, and apple placement), our method raises the average completion score from 50.8% to 65.5%, while adding 326 ms for sampling and verification per triggered decision on a single RTX 4090. Consistent gains are also observed in simulation, where the average success rate rises from 66.4% to 73.6% in the clean setting of RoboTwin 2.0, from 62.8% to 69.6% in its randomized setting, and from 96.9% to 97.6% on LIBERO.

Thu 15 OctRobotics
The gist
Robotic systems that decide actions by sampling can sometimes pick a poor choice, which can cause tasks like inserting a pen or placing an apple to fail. The authors designed a way to check action quality during execution by comparing multiple candidate actions and predicting their effects in a simplified internal model. This step helps the robot choose better actions without retraining, leading to more successful task completion in both real and simulated tests.
Open → 2610.21985v1

RoboSpan enables better coordination for whole-body robot tasks

RoboSpan: Learning Modular Coordination Policies for High-DoF Whole-Body Mobile Manipulation with Teleoperation

Abstract: Whole-body mobile manipulation suffers from two fundamental challenges: inherent motion heterogeneity across locomotion, arm, and hand modules, and progressive error accumulation over long-horizon multi-stage tasks. Existing teleoperation platforms also lack integrated dexterous tactile sensing to support coordinated whole-body decision-making. This paper presents RoboSpan, a high-DoF whole-body teleoperation and demonstration platform, and the Hierarchical Modular Framework (HMF) for learning modular coordination policies. HMF functionally decouples the 29-DoF action space into fine hand manipulation, arm pose adjustment, and mobile-base spatial configuration branches, and introduces the Coordinated Modular Embodied Transformer (COMET) with inter-modular cross-attention to enable dynamic information exchange and action coordination. Vision-language model driven hierarchical task decomposition structurally suppresses temporal error propagation, while a full-palm 2D tactile representation encodes distributed contact signals for cross-module tactile empowerment. Experiments on a physical 29-DoF robot demonstrate that the proposed method achieves over 20\% higher task success rate than mainstream baselines across a range of long-horizon whole-body tasks, markedly improving stability and robustness in contact-rich manipulation.

Wed 14 OctRobotics
The gist
Robots that move their whole body to complete tasks face challenges because their legs, arms, and hands all need to work together smoothly. The authors created RoboSpan, a system that breaks down these complex movements into smaller parts and uses a special method to coordinate them. Their approach also includes touch sensors on the robot’s hands to help with delicate tasks. Tests show RoboSpan helps robots do complicated, multi-step jobs more successfully and with better control.
Open → 2610.19594v1

Robot hand design improves copying human finger movements

PACT: Posture-Aligned Co-Design Technique for Hand Configuration and Retargeting in Multi-Finger Imitation Learning

Abstract: The fidelity of teleoperated demonstrations used for imitation learning is determined by how faithfully a robot hand can replicate the operator's posture. The posture retargeting law that the robot will execute is typically selected only once the hardware has been finalized, and conventional hand designs often neglect human-like grasping postures. We introduce PACT, an optimization based on human grasping postures that jointly optimizes the mechanical parameters of a 17-servo robot hand (link lengths, joint offsets, mounting angles, and base placements) and its nonlinear posture retargeter (a linear human-to-robot joint map, per-joint posture-IK weights, and damping). The optimization enforces full-finger correspondence (MCP, PIP, DIP, and tip of each finger) to human poses from the DexYCB dataset, and is constrained by servo interference, assembly clearances, and flat-hand protrusion, so the resulting design can be physically assembled. We fabricate the PACT Hand and validate it through teleoperation experiments and an imitation-learning demonstration. The hand reproduces all human hand keypoints with a kinematic RMSE of 3.9 mm, compared to 12.8 mm for the LEAP Hand baseline, and keeps all fingertips within 10 mm on every grasping posture, while the baseline achieves this for only 21.6% of the poses. Under matched per-joint torque limits, it lifts twice the baseline payload, 2 kg against 1 kg, and deforms a held paper cup seven times less, reaching a deformation level comparable to a human hand. It also achieves an 86% grasping success rate on YCB daily objects, compared to 64% for the baseline. Finally, as a demonstration, an imitation-learning policy trained on demonstrations collected with the PACT Hand achieved a 75% success rate at held-out placements in a can pick-and-place task. Project page: https://pact-hand-2027.github.io

Tue 13 OctRobotics
The gist
Teleoperated robot hands often struggle to copy human hand poses accurately, limiting how well they can learn from humans. The authors developed a technique called PACT that jointly designs a robot hand’s mechanical structure and its way of translating human hand movements into robot movements. By optimizing both together using real human grasp data, they built a 17-motor robot hand that matches human finger positions much more closely than a baseline design. This hand also performs better at picking up objects, handling delicate items, and learning tasks from demonstrations.
Open → 2610.18443v1

AutoBio plus enables robot testing in realistic biology labs

AutoBio+: Benchmarking Dexterous Manipulation with Dynamic Experimental Phenomena in Realistic Laboratories

Abstract: Automating biological laboratories requires robots to perform dexterous tool interactions under realistic visual conditions while handling dynamic experimental phenomena, presenting significant challenges for robotic manipulation. To enable systematic evaluation under these conditions, we introduce AutoBio+, a simulation benchmark and framework for dexterous manipulation in realistic biological laboratory environments. AutoBio+ comprises 14 tasks across 4 robot embodiments, spanning functional tool use, precision manipulation, language-conditioned visual grounding, and instrument operation. We introduce controllable dynamic experimental phenomena, including flames, vapor, and bubbles, to capture the evolving visual conditions of laboratory procedures. To further enhance environmental realism, we develop a hybrid rendering pipeline that integrates interactive simulated foreground objects with Gaussian-splatting backgrounds reconstructed from 6 real laboratory scenes. For efficient large-scale data generation and evaluation, we further build a parallel simulation framework on top of mjlab, proposing Parallel Finite-State Machine (PFSM) and integrating batched rendering with decoupled trajectory replay for demonstration generation and model-independent closed-loop policy evaluation. Our evaluation reveals substantial variation in policy performance across tasks, exposing distinct strengths and weaknesses of current manipulation policies. Real-world experiments further demonstrate a reduced sim-to-real gap, supporting the validity of AutoBio+ for realistic manipulation evaluation. Supplementary materials are available at https://autobio-plus.github.io/AutoBioPlus.

Fri 9 OctRobotics
The gist
Robots need to handle delicate tools and changing environments in biology labs, which is very hard. The authors created AutoBio+, a detailed simulation system where robots practice 14 tasks in realistic lab conditions, including flames and bubbles. This helps test and compare robot skills reliably, showing which robots do better or worse on different tasks. They also show their simulation matches real lab conditions well, making it useful for developing real robots in biology.
Open → 2610.13643v1

Jointly retarget human and object motion for humanoid robots

OTRetarget: Joint Robot and Object Motion Retargeting via Optimal Transport

Abstract: Transferring human motion to humanoid robots requires adapting the demonstrated motion to the robot morphology while preserving interactions with the environment. This is particularly challenging for loco-manipulation tasks, where contacts with the ground and manipulated objects must remain consistent despite differences in body proportions. Yet, skeletal motion alone does not fully describe these interactions, and fixing object trajectories limits the adaptation to a new embodiment. In this paper, we introduce OTR ETARGET, a unified approach to jointly retarget robot and multi-object motion from human demonstrations. Our approach represents surface interactions through signed distances, closest surface points, and relative directions, and uses entropic optimal transport to transfer these quantities across human, robot, and object geometries. We incorporate the resulting interaction targets into a constrained inverse kinematics formulation that balances contact preservation with motion style and jointly optimizes robot and object poses at each frame. This formulation accommodates robot-object and object-object interactions without rescaling the scene or the demonstration. We validate the proposed approach on OMOMO, where it achieves a robot- object interaction Jaccard score of 87% and a depth error of 8.7 mm, compared with 28% and 29.3 mm for OmniRetarget. Finally, we demonstrate transfer to a physical G1 humanoid using whole-body policies trained with reinforcement learning on the retargeted references, across motions including two-handed box pick-and-place onto a table.

Tue 29 SeptRobotics
The gist
Turning human actions into movements a robot can perform is tricky, especially when the robot needs to interact with objects and the environment just like a human would. The authors created a method that copies both the robot’s body movements and the objects being moved simultaneously, keeping interactions accurate despite differences in size and shape. Their approach uses math to match surfaces and directions between humans, robots, and objects, then solves how the robot and objects should move together at every moment. Tests show this method works better than a previous one and can control a real robot doing tasks like moving boxes with both hands.
Open → 2609.36602v1

X-Reset trains robot hands to grasp diverse objects using human resets

X-Reset: Scaling Object-Centric Reinforcement Learning via Cross-Embodiment Resets

Abstract: Reinforcement learning (RL) in simulation can train dexterous manipulation policies without robot demonstrations, but training a single generalist policy with task-agnostic rewards faces a severe exploration problem: approaching, grasping, and reorienting diverse objects with many degrees of freedom is difficult to discover from scratch. Prior works make exploration tractable with high-quality robot demonstrations, per-task reward shaping, or by restricting policies to narrow modes of behavior. We propose X-Reset, a framework that instead resolves exploration with human hand-object demonstrations. Rather than imitating or tracking retargeted human motion, X-Reset kinematically retargets hand-object states to noisy robot states, filters out states that are unstable in simulation, and samples the remainder as resets during RL training with general-purpose object-centric rewards. The resulting policy depends only on object state and goal, with demonstrations entering training through the reset distribution. We show that X-Reset trains generalist policies on 20 objects across three embodiments---a 22-DoF hand on two different arms and a parallel-jaw gripper---and resolves the exploration challenges of RL from scratch. X-Reset scales with the number of training objects, generalizes to unseen objects, can learn from imperfect hand-pose estimates, and transfers behaviors zero-shot from sim-to-real.

Mon 28 SeptMachine LearningArtificial IntelligenceRobotics
The gist
Training robot hands to pick up and use many different objects is hard because robots struggle to learn these skills from scratch. The authors propose X-Reset, a method that uses snapshots from human hand motions interacting with objects to help robots start their learning in helpful positions. Instead of copying human movements exactly, X-Reset converts these human poses into robot poses and carefully chooses stable states to reset to during training. This approach allows robots to learn general skills for many objects across different types of robot hands, even working with imperfect human data and transferring skills from simulation to real robots.
Open → 2609.35715v1

Collocated control stabilizes shape regulation in soft robots

Collocated Shape Regulation for Soft Robots

Abstract: Controlling the shape of a continuum soft robot typically requires an accurate dynamic model and actuation of all degrees of freedom. We show that regulating only the actuated coordinates, through collocated shape control, achieves provably stable convergence of those coordinates and, under an explicit compatibility condition, of the entire robot shape. While collocated control is a cornerstone of high-performance motion control in rigid robotics, extending this formulation to continuum soft robots has remained challenging due to the complexity of their dynamics. We present the first general framework for collocated control of continuum soft robots and derive a unified family of controllers, including PD, PID, PsatID, and their counterparts with compensation and cancellation components. The framework unifies existing approaches while introducing new controller designs. In particular, we develop three classes of PD and PID like regulators with local, semi-global, and global stability guarantees, and provide rigorous convergence analyses for each. Extensive experimental validation demonstrates the effectiveness of the proposed methods across different model discretizations and controller parameters. The resulting framework provides practical design guidelines for selecting and implementing controllers with known stability guarantees, without requiring a complete dynamic model of the robot

Wed 23 SeptRobotics
The gist
Controlling the exact shape of flexible soft robots is usually complicated and needs precise models and controlling every part. The authors show that by controlling only the parts directly moved by motors or actuators, it’s possible to reliably shape the whole robot under certain conditions. They developed a general approach with several control methods that ensure stable and predictable robot shapes without knowing the full robot dynamics. Their tests confirm this method works well with different robot setups and control settings. This simplifies designing shape control for soft robots while keeping stability.
Open → 2609.27469v1