Papers for

robotics integration teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Autonomous system creates detailed human assembly plans from CAD designs

End-to-End Autonomous Generation of Human Assembly Plans

Abstract: Turning a CAD design into an assembly plan is still largely done by hand, requiring engineers to reason about geometric feasibility, tool access, stability, and the ergonomics of human assembly. In this work, we encode long-established design for assembly (DfA) principles into a contained, end-to-end approach for generating assembly plans. Our approach takes only a mesh assembly and produces either a step-by-step assembly manual or a structured failure report, requiring no joint metadata, fastener annotations, or additional information. Four major components of a manufacturing plan are addressed autonomously: an assembly tool list, the assembly sequence and subassemblies, an assembly manual, and design feedback for improving assemblability. For determining the sequence plan, we systematically disassemble the object in a physics simulator and apply a cost function that encodes DfA principles. Manual generation, tool labelling, and assembly feedback rely primarily on multimodal large language models. Compared with a baseline that always removes the outermost part first from Tian et al., DfA-aware sequence planning reduces simulated assembly time, measured with a robot-arm assembly-time proxy, by 35% on 136 assemblies of 5 to 30 parts. The correct tool is selected for 88.6% of assembly steps. A vision-language model judge compares the generated manuals against ablated variants, identifying which page elements carry the information a reader needs. The presented approach and open-source code are available for use by engineers or AI agents looking to rapidly accelerate the creation of manufacturing plans for a given product design.

Wed 7 OctRoboticsGraphics
The gist
Making step-by-step assembly instructions from a computer design still usually requires engineers to carefully plan how parts fit together and how people will assemble them safely and efficiently. The authors developed an all-in-one method that takes only a 3D model made of parts and automatically generates a full assembly guide or a report explaining why assembly might fail. Their system uses a physics simulator to figure out the best order to assemble parts and AI language models to write manuals and choose tools. This approach can reduce robot-simulated assembly time by 35% and selects the right tools about 89% of the time, helping engineers speed up manufacturing preparation.
Open → 2610.09781v1

Robots learn to fix pose errors for more precise assembly tasks

Seeing Through the Displaced Frame: Privileged Noise Distillation for Vision-Force Precision Assembly

Abstract: Pose error in precision assembly can corrupt not only what a robot observes but also the coordinate frame in which it acts. On the FORGE benchmark, the official state-based policy succeeds in 97% to 99% of episodes with the true pose but only 32% to 60% at the benchmark's $σ=5$ mm pose-noise setting. The same estimated pose enters the observation and anchors the action frame, making the offset unidentifiable from proprioceptive state alone before contact. We supply this missing information during training in two ways. A privileged teacher observes the offset in simulation, while clean demonstrations can instead be relabelled into the displaced frame in closed form. The deployed student is trained with behaviour cloning followed by one DAgger round and receives only noisy state, a raw wrench window, and two RGB cameras at test time. On the unmodified FORGE tasks, the teacher-route student maintains 92% to 99% success across $σ=0$ to 5 mm, while six non-privileged baselines fall to 2% to 80%. A matched behaviour-cloning experiment isolates the source of this robustness. With the same student architecture, data budget, and training procedure, demonstrations generated without offset access yield only 31.5% success at $σ=5$ mm, whereas privileged and relabelled demonstrations reach 88.7% and 95.8%. Deployed zero-shot on a Franka, the student reaches 83.3% pooled success at $σ=5$ mm against 34.4% for the strongest state-based policy. Deployable sensing alone is insufficient. Robustness requires supervision that encodes compensation for the latent frame offset.Project page: https://drychang.github.io/displaced-frame/

Tue 6 OctRobotics
The gist
Robots often struggle with small errors in how they see or hold parts during precise assembly, causing failures. The authors developed training methods where a special teacher system knows the exact error and helps the robot learn to adjust for it. This approach lets robots keep high success rates even when their sensors are noisy or inaccurate. They tested this on a robot arm and real tasks, showing big improvements over methods without this help.
Open → 2610.07745v1

ForceTwin identifies precise physics for better robot object handling

ForceTwin: Physics-informed Digital Twins for Robotic Manipulation from Instrumented Human Interaction

Abstract: Manipulating objects requires understanding not only their motion, but also the physical properties that determine it. For articulated objects, these include inertia, friction, and mechanisms such as springs or door closers, whose effects can vary with configuration and velocity. Such properties are not directly observable from appearance: visually identical doors may require very different effort to manipulate. Existing digital-twin pipelines recover primarily kinematics or assign static physical parameters from visual and language priors, which can yield physically implausible estimates. As a result, state-dependent mechanism dynamics remain unidentified and are not represented in standard asset formats. We present ForceTwin, a system for identifying physics-informed digital twins of articulated objects from instrumented human interaction. A person probes an object using a handheld force-sensing gripper, providing synchronized poses and interaction forces from which we estimate the articulation, parametric dynamics including inertia, Coulomb friction, viscous damping, and a structured neural residual capturing state-dependent mechanism forces. ForceTwin nearly halves the inertial-parameter error of a VLM prior. As a feedforward dynamics model for impedance control on a Spot and a Franka FR3, ForceTwin achieves 87% goal completion across nine object-embodiment pairs, compared with 60% using VLM-prior and 57% using kinematics-only twins, with the largest gains on objects whose strong mechanisms cause both baselines to stall. We further use the identified twins to train whole-body door-traversal policies and deploy them in the real world. Project Page: https://timengelbracht.github.io/forcetwin-website/

Fri 18 SeptRoboticsArtificial Intelligence
The gist
Robots have trouble opening or moving objects like doors because they don’t know the forces involved, such as friction or spring effects. The authors created ForceTwin, which watches a person use a special tool to measure force and movement on these objects. This data helps build a detailed, physics-based model so robots can predict how objects will behave. Using these models, robots can manipulate objects more successfully, especially ones with tricky mechanisms.
Open → 2609.21751v1

UAV navigation aligns language commands with diverse safe flight paths

VLM-MPPI: Grounding Natural Language in Behaviorally Diverse Trajectories for Aerial Navigation

Abstract: We present a hierarchical UAV navigation framework that aligns natural-language intent with dynamically feasible flight behaviors in cluttered indoor environments. To bridge the gap between abstract semantics and low-level control, we employ a parallelized ensemble of six behavior-conditioned Model Predictive Path Integral (MPPI) planners. Crucially, by designing mode-specific guiding costs and sampling biases, we induce distinct trajectory modes that converge to unique behavioral means, yielding a compact set of intentionally diverse candidates rather than mere stochastic variations. We project these 3D candidates onto the onboard first-person-view RGB stream, turning language grounding into a visual action selection problem. A pretrained vision--language model (VLM) asynchronously selects the candidate index given the overlaid FPV image and a natural-language prompt, while MPPI replans at 20Hz and a PID-based low-level controller tracks the selected trajectory. We implement the full pipeline in NVIDIA Isaac Sim and on a real-world quadrotor platform equipped with LiDAR and RGB sensing. Experiments in both simulation and real-world flights show semantically meaningful behavior diversity, robust language alignment despite VLM latency, and safe, repeatable flight across all modes, achieving 100% task success in our evaluated scenarios.

Wed 16 SeptRobotics
The gist
Flying drones indoors is tricky because they need to avoid obstacles while following instructions. The authors created a system that plans six different safe flight paths based on what the drone is told to do. They turn the problem of understanding language into picking the best flight path by matching pictures the drone sees with the instruction using a visual-language AI model. Their system works well both in computer simulation and on a real drone, flying safely and successfully every time.
Open → 2609.18451v1

Large language models steady networked control systems with slow supervision

Large Language Models in the Loop: A Stability- and Network-Aware Survey in Networked Control, Cyber-Physical, and Multi-Agent Systems

Abstract: Modern networked control systems (NCSs), cyber-physical systems (CPSs), and complex multi-agent network systems (CNSs) increasingly rely on large language models (LLMs) for high-level decision-making. However, the slow, stochastic nature of LLMs directly conflicts with the strict stability and safety guarantees required by these physical systems. This survey presents a unified analysis of how LLMs can be admitted into the control loop of NCS, CPS, and CNS without compromising closed-loop guarantees. We organize this around a core principle: the LLM operates as a slow supervisor adjusting high-level goals and constraints, while a fast, certified inner loop maintains physical stability. Under this framework, LLM integration maps directly to classical networked control challenges, where inference latency acts as delay, API failures as packet dropouts, tokenization as quantization, and hallucinations as bounded disturbances. We assess current developments across all these three domains, highlighting that rising model capabilities are frequently accompanied by a drop in formal safety assurances. Finally, we propose concrete future research directions, identifying the widespread lack of formal stability proofs as the field's central open problem.

Tue 15 SeptArtificial Intelligence
The gist
Some computer systems that control physical devices or multiple agents need to be very reliable and stable. The paper's authors review how large language models (LLMs), which usually take time to respond and can be unpredictable, can still be included in these systems safely. They explain that if the LLM acts slowly to set high-level goals while a faster, certified system handles immediate control, stability can be maintained. They also compare typical LLM behaviors to known network control problems and say more research is needed to guarantee safety formally.
Open → 2609.16599v1