Papers for

robotic simulation developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Vision language action models overcome robotic grasp failures with diagnostic guidance

Breaking the Average: Overcoming Convergent Defaults in Robotic Manipulation via Diagnostic Chain-of-Thought and Structured Guidance

Abstract: Recent Vision-Language-Action (VLA) models, such as $π_{0.5}$, have demonstrated impressive generalization capabilities by scaling behavior cloning across massive datasets. However, these models frequently suffer from distributional convergence, a phenomenon where the policy regresses to "safe" but suboptimal mean behaviors due to mode averaging in the training data. This often results in generic grasp strategies, Convergent Defaults, that fail in constrained or long-tail scenarios. In this work, we propose a novel framework to identify and correct these behaviors without parameter updates. We introduce a three-stage methodology: (1) Probing for Defaults, identifying task-specific mode collapse; (2) Diagnostic Root Cause Analysis, utilizing the VLA's latent reasoning capabilities to verbalize the logic behind suboptimal actions; and (3) Structured Guidance Injection, where concrete, distinct alternatives are encoded into reusable "Lens Prompts" that shift the sampling distribution. We validate this approach using $π_{0.5}$ in ManiSkill3 simulation environments. We hypothesize that our method outperforms vanilla $π_{0.5}$ and standard Chain-of-Thought prompting by effectively breaking convergent defaults in complex manipulation tasks.

Tue 13 OctRobotics
The gist
Robotic systems that combine vision and language understanding sometimes learn average or safe ways to grasp objects, which can fail in tricky situations. The authors show how to spot these average behaviors and use the system's internal reasoning to explain why they happen. Then, they introduce new guided instructions that help the robot try different, more effective grasps without changing the robot’s original learning. This method helps robots do better at complex tasks.
Open → 2610.17983v1

Usdcraft builds realistic robot parts from partial 3d scans

USDCraft: Geometrically Grounded Programmatic Modeling of Articulated 3D Assets for Simulation

Abstract: Geometrically faithful and functional articulated 3D assets are essential for real-to-sim robot manipulation, where policies trained in simulation must transfer to physical objects. Recent mesh-based methods learn to infer articulation from annotated 3D assets, but deployment remains challenging when real-world objects fall outside the training distribution or their meshes are incomplete or corrupted. To address these limitations, we formulate articulated asset reconstruction as programmatic modeling grounded in partial geometric evidence and introduce USDCraft, a framework in which a pretrained LLM writes and revises executable programs for simulation-ready articulated assets without task-specific training. We propose source geometry analysis, which converts the source mesh into a metric textual description that distinguishes observed surface from unknown space, and iterative geometric rechecking, which re-encodes each candidate in the same representation so that discrepancies point to program edits while unobserved regions remain open to completion. Visual feedback and physical authoring guidance complete the modeling process, which produces articulated USD assets with explicit physical properties that load into Isaac Sim without manual adjustment. Experiments demonstrate leading articulation recovery on two benchmarks and validate USDCraft's effectiveness for real-to-sim-to-real robot manipulation.

Thu 8 OctRobotics
The gist
Robots often learn how to handle objects by practicing in computer simulations, but making these virtual objects work like real ones is tricky when the 3D scans of those objects are incomplete or imperfect. The authors present USDCraft, a system that uses a large language model to write computer programs that rebuild these objects with moving parts based on partial 3D data, without needing to be specially trained for each task. Their approach checks and improves the program step-by-step by comparing it with the scanned data, allowing it to fill in missing parts and produce simulation-ready assets automatically. This helps robots better transfer what they learn in simulations to the real world.
Open → 2610.11322v1

History-based model predicts uncertain motions of soft robotic arms

History-Conditioned Flow Matching for Probabilistic Dynamics of Tendon-Driven Continuum Robots

Abstract: Deterministic dynamics modeling of tendon-driven continuum robots remains challenging owing to uncertainties in material behavior, tendon transmission, friction, and contact. Measured joint configurations and nominal tendon commands do not fully characterize these internal mechanical factors, leaving uncertainty in the subsequent motion. We therefore develop a history-conditioned, physics-informed flow-matching framework for probabilistic dynamics prediction, using motion and actuation histories to predict the distribution of the next complete joint configuration. By recursively sampling next-step configurations under prescribed commands, the model predicts distributions of future whole-body motions. In simulation, scenario-specific models achieve five-second trajectory Energy Scores (lower is better) of 12.05 mm under internal friction variation and 9.29 mm under unobserved actuation disturbances. Relative to the conditional variational autoencoder and diffusion baselines, Flow attains lower Energy Scores and coverage closer to the nominal level in both scenarios. Ablations support history and structural conditioning in both scenarios. On the physical robot, predictions under two tendon-command profiles excluded from training capture the principal motion sequences, with five-second Energy Scores of 11.91 and 11.42 mm, lower than the compared baselines. The predicted-to-measured spread ratios are 1.65 and 1.22 (closer to 1 is better). These results support history-conditioned probabilistic dynamics prediction under incomplete mechanical observations.

Tue 22 SeptRobotics
The gist
It is difficult to predict how soft robotic arms controlled by tendons will move because of unknown factors like friction and material differences. The authors introduce a way to learn from past movements and commands to better estimate all possible next positions of the robot’s joints. Their method produces accurate predictions in computer simulations and tests on a real robot, capturing the typical sequences of motion even when some conditions change. This helps to understand and control tendon-driven robots despite unseen internal variations.
Open → 2609.25658v1