Papers for

autonomous drone operators

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

GeoReVis improves remote sensing image segmentation by combining mask revisions

GeoReVis: Learning Selective Revision and Evidence Consolidation for Remote Sensing Segmentation

Abstract: Recent methods try to inspect and revise masks to mitigate referential ambiguity and localization errors caused by complex spatial relations in language-guided remote sensing segmentation. However, these solutions often lead to Revision Degradation: revising missing regions may introduce errors into previously correct ones. Even a globally better mask can omit correct regions found in inferior masks. We term this Evidence Fragmentation: correct target evidence is scattered across imperfect masks. All masks generated by the model at this time are suboptimal. We propose GeoReVis, which unifies revision guided by execution feedback, Candidate Selection, and Cross-Mask Evidence Consolidation. It learns fine-grained revision from real actions that improve segmentation when executed and learns to select a better mask through quality comparisons between actual candidates, reducing the risk of degradation during revision. The selected whole mask can still discard useful regions. GeoReVis therefore retains all candidates across rounds, partitions their spatial disagreements into composable regions, and jointly reconstructs regions conditioned on the full instruction and context. This makes complementary and fragmentary evidence reusable regardless of its source candidate's overall quality, yielding improved segmentation. We further build a data pipeline to generate fine-grained multi-round segmentation trajectories and derive supervision for revision actions, candidate selection, and region composition from the model's own outputs, extending training beyond whole-mask IoU supervision. We also propose ReVis-Eval, a benchmark protocol evaluating multi-round revision and cross-round evidence consolidation accuracy. Experiments on ReVis-Eval and 5 benchmarks for remote sensing reasoning and referring segmentation demonstrate the effectiveness of GeoReVis.

Sun 15 NovComputer Vision and Pattern Recognition
The gist
Segmenting images from satellites or drones can be tricky because shapes and regions can be unclear or overlapping. The paper finds that trying to fix mistakes by revising segmentation masks can actually cause new errors or lose correct parts. The authors created GeoReVis, a method that carefully chooses better masks and combines pieces from multiple tries to keep all correct information together. This approach helps get more accurate segmentations by learning from past corrections and mixing evidence from different attempts.
Open → 2611.20183v1

Robots share task-focused views to improve teamwork in limited communication

Be My Eye: Learning to See Beyond a Robot's Own Perspective

Abstract: Robots operating in the real world are inherently subject to partial observability, particularly in multi-robot collaboration, where individual observations may be insufficient for effective task execution. A key challenge is leveraging complementary observations from partner robots to support collaborative decision-making. We argue that information sharing should go beyond passively transmitting raw observations or generic visual features. Instead, robots should selectively encode task-relevant information according to task objectives and execution progress. To this end, we propose a task-oriented active context sharing framework that transforms robot observations into compact representations, preserving critical information under constrained communication budgets. The framework adopts a two-stage training paradigm. In the first stage, onboard observations and task instructions are jointly encoded into compact context representations. Task-oriented future prediction and soft task-progress correspondence learning guide this collaboration-agnostic compression, preserving information about task objectives and execution progress. In the second stage, the learned representations are integrated into a pretrained Vision-Language-Action (VLA) policy, enabling bidirectional context exchange. Each robot combines complementary partner information with local observations to guide action generation. Action supervision jointly refines the context representations and policy, aligning information sharing with collaborative decision-making. Controlled experiments demonstrate substantial performance gains under constrained communication budgets, while three real-world experiments validate the framework in complex dual-robot tasks. These results highlight a shift from passive visual transmission toward active, task-oriented context sharing that directly supports collaborative decision-making.

Mon 9 NovRoboticsArtificial Intelligence
The gist
Robots working together often can't see everything they need because they each have their own limited viewpoint. The authors created a way for robots to share only the most important, task-related pieces of what they see so they can make better decisions together. Their system teaches robots to compress their observations into small messages that still contain key info about the task and progress. This helps pairs of robots work more effectively even when they can only send small amounts of data to each other. They tested this on real robots completing tricky tasks and saw better cooperation.
Open → 2611.12295v1

Event cameras improve spacecraft pose accuracy with hybrid methods

Direct Regression and Geometric Recovery for Event-based Spacecraft Pose Estimation

Abstract: Event cameras offer high temporal resolution and dynamic range for uncooperative spacecraft pose estimation, but models trained on synthetic event streams must generalize to the noise, background activity, and contrast variation of physical sensors. We present a comparative systems study of two independently developed approaches to synthetic-to-real, event-based 6-DoF spacecraft pose estimation. The first is a modular pipeline that combines learned keypoint detection and unlabeled target-domain adaptation with geometry-based pose recovery and temporal smoothing. The second directly regresses translation and rotation from global and cropped event representations using an apparent-scale cue, physics-motivated synthetic corruptions, and feature-consistency regularization. We evaluate both systems under the same SPARK 2026 hidden-test protocol on the real-event split of SPADES, with predictions for all test frames. The results reveal a component-wise reversal concealed by the aggregate challenge score: direct regression reduces orientation error by 22.7% (1.1767 to 0.9101 rad) and obtains a 13.4% lower composite score, whereas the modular pipeline reduces relative translation error by 78.7% (0.1335 to 0.0285). Archived configurations and a qualitative failure case identify plausible explanations for these complementary error profiles, without isolating their causes. These findings motivate a testable hybrid design that combines direct orientation estimation with geometry-constrained translation recovery.

Fri 6 NovComputer Vision and Pattern Recognition
The gist
Figuring out exactly where a spacecraft is and how it's oriented using fast event cameras is tricky because real sensor data is noisy and different from simulated data. The authors compare two strategies: one breaks down the problem into parts, using key points and geometric math; the other predicts position and rotation directly from the sensor data. Their tests show each method has strengths—one is better at knowing how far the spacecraft is, the other at its direction. They suggest combining these approaches to get the best of both worlds.
Open → 2611.09717v1

Multispectral aerial object detection improves with global local dynamic modeling

SGLDet: Spectral-aware Global-Local Dynamic Modeling for Multispectral Aerial Object Detection

Abstract: Multispectral aerial object detection aims to exploit spectral-spatial cues to address object detection challenges in complex aerial scenes. Existing methods mainly focus on spectral-spatial decoupling and object-aware spectral-spatial fusion to improve spectral utilization. However, two critical issues remain underexplored: 1) severe foreground-background imbalance caused by large scene coverage hinders reliable object recognition; 2) aerial objects exhibit substantial local variations in scale, orientation, and structure, which hinder accurate localization. To address these issues, we propose SGLDet, a spectral-aware global-local dynamic modeling framework for multispectral aerial object detection, which is motivated by the observation that object semantics are jointly determined by global spectral consistency and scene context, while precise localization requires dynamic feature modulation conditioned on the input spectral and spatial properties. To realize these properties, SGLDet consists of three coordinated components: a Spectral-driven Global Dynamic Modeling module that aggregates spectrally related features across the scene via similarity-guided sparse matching; a Multi-scale Global Context Integration module that constructs a compact scene memory for scene-guided contextual reasoning; and a Spectral-aware Input-dependent Calibration Aggregation module that performs expert-kernel aggregation with scene-guided spectral-spatial calibration to enable adaptive local feature modulation. Extensive experiments on MODA, HOD3K, and DroneVehicle demonstrate that SGLDet achieves superior performance with competitive computational cost. Code is available at https://github.com/Riiluo/SGLDet.

Tue 20 OctComputer Vision and Pattern Recognition
The gist
Detecting objects in aerial images taken with multiple spectral bands is hard because objects can vary a lot and there is often much background clutter. The authors of this paper designed a new system called SGLDet that looks at both the whole scene and local object details using spectral information to spot objects more accurately. Their approach also adjusts its detection strategies based on the unique features of each scene. Testing on several aerial image datasets showed that SGLDet performs better than prior methods while keeping computation demands reasonable.
Open → 2610.27883v1

Robot adapts thinking time to improve real-time control

Learning Robot Metareasoning: Adaptive Computation Budgets for Real-Time Control

Abstract: Planning while moving faces an historical trade-off between deliberation and reactivity. A planner running concurrently with execution needs time to compute, so the robot must commit to an open-loop action chunk for that duration: a longer commitment buys a better plan at the cost of feedback frequency. Existing controllers fix this action commitment, and consequently the computation budget, before deployment. However, the optimal computation time is highly state-dependent (due to factors such as unmodeled dynamics, safety constraints, and marginal optimisation gains), i.e. dynamic, therefore offline static tuning risks leaving systems blind during critical maneuvers or unable to find global effective solutions. We cast the identification of the optimal computation budget as a semi-Markov decision process (SMDP) in which a single decision defines both the robot action commitments and the time allocated to the solver. To solve this, we train a reinforcement-learning policy conditioned on the current state, the executing plan, and inexpensive planner statistics. Across ground-robot navigation, agile quadrotor flight, and contact-rich manipulation, the learned policy changes its budget with the demands of the scenario, discovering efficient reasoning patterns that outperform fixed-budget baselines. https://lorenzomagnino.github.io/learning-robot-metareasoning-review/

Fri 16 OctRobotics
The gist
Planning robot actions requires balancing between thinking time to choose good actions and reacting quickly to changes. The authors show that instead of fixing how long a robot spends planning, the robot can learn when to plan longer or shorter based on the situation it faces. They teach the robot this by using reinforcement learning, which lets the robot figure out the best balance in real time. Their tests on robots that navigate, fly, and handle objects show this method works better than using a fixed planning time.
Open → 2610.23662v1

Fpv-crash dataset locates exact moments drones crash in flight videos

FPV-CRASH: A Dataset for Offline Crash Moment Localization in Drone First-Person-View Videos

Abstract: Crash Moment Localization aims to localize the major physical impact that interrupts controlled flight in an onboard first-person-view (FPV) video recorded by a drone, thereby providing a temporal reference for retrospective crash analysis in aerial robotics. However, existing crash-video datasets predominantly target surveillance or dash-camera footage, and their annotation schemes and task formulations are not designed for temporal localization of the major physical impact in UAV onboard FPV videos. To address this gap, we present FPV-CRASH, a dataset and fixed-protocol reference evaluation for offline Crash Moment Localization in drone FPV videos. FPV-CRASH contains 1,052 drone FPV clips (526 Crash and 526 Normal); each Crash clip has a single-frame reference annotation for the Crash Moment, and every clip is paired with four English captions automatically generated from sampled frames and the known Crash/Normal label. Normal clips carry no timestamp target and provide training-time background supervision. The captions support an auxiliary Text-to-Video Retrieval task, whereas localization inference remains video-only. We further provide a heatmap-based reference model that estimates a continuous Crash Moment timestamp from sparsely sampled frames. Under the fixed, non-source-disjoint clip-level protocol, the reference model reduces MAE from 1.12 s for the strongest fixed timing-prior baseline to 0.56 s (a 50% relative reduction) and achieves a median absolute error (MedAE) of 0.33 s; the fraction of clips localized within 0.5 s rises from 37.14% to 65.71%. The release includes annotations, available source video identifiers and URLs, clipping metadata, fixed split lists, and preprocessing code so that the reportable portion can be reconstructed without redistributing source footage. At submission time, all source-compilation URLs remained accessible.

Wed 14 OctComputer Vision and Pattern Recognition
The gist
Drones can crash while flying, and figuring out the exact moment a crash happens in drone video footage is important for studying what went wrong. The authors created FPV-CRASH, a collection of over 1,000 drone flight videos that show both crashes and normal flights, with labels marking the exact crash moment. They also developed a model to predict when a crash happens from the video frames, improving accuracy by about half a second. This helps engineers analyze crashes more precisely using the drone's own video recordings.
Open → 2610.19785v1

Reinforcement learning improves safe navigation in unknown environments

RL-Guided PAC-NMPC for Probabilistically-Safe Perception-Based Navigation in Unknown Environments

Abstract: In this paper, we present an approach for combining stochastic nonlinear model predictive control (SNMPC) and reinforcement learning (RL) to enable probabilistically-safe perception-based navigation in unknown environments. Our method first uses RL to train probabilistic actor-critic and sensor prediction models. We then leverage these probabilistic models in a sampling-based SNMPC framework known as Probably Approximately Correct (PAC)-NMPC, which uses hard constraints to enforce finite-time statistical guarantees on the probability of collision and value function improvement. By ensuring that our finite-horizon SNMPC policies decrease the value function in expectation, we can approach the long-horizon performance of the RL approach while satisfying probabilistic safety constraints. Through simulation experiments, we show that our approach can improve the safety of perception-based RL navigation policies and scale to high dimensional systems with large sensor input spaces and complex nonlinear dynamics. We also demonstrate our approach through hardware experiments, showing improved performance for vision-based navigation with an agile fixed-wing aerial vehicle in unknown environments.

Wed 30 SeptRobotics
The gist
Navigating safely through unknown places is tricky because robots must avoid collisions while figuring out where to go. The authors combine two techniques: reinforcement learning, which helps robots learn from experience, and a special planning method that ensures safety with high probability. Their approach trains models to predict sensor data and control actions, then plans moves that reduce risks while aiming for good outcomes. They tested this both in simulations and with a flying robot, showing improved safety and better navigation in complex settings.
Open → 2609.39854v1

Cross modal tracking improved by aligning template and search features

Template-Search Domain Adaptation via Multi-Stage Feature Alignment for Cross-Modal Object Tracking

Abstract: Visual object tracking typically assumes that the initial template and subsequent search frames share the same sensing modality. In practice, sensor availability or operation may change over time, creating a substantial representation gap between template and search frames. Unlike conventional multi-modal tracking where paired modalities are simultaneously available, cross-modal tracking requires localization when template and search frames originate from different active modalities. Accordingly, we introduce TSDA-Track, a Template-Search Domain Adaptation framework to reduce modality discrepancy during training. We investigate two feature alignment strategies. Pre-AFA TSDA-Track applies adversarial alignment before transformer's template-search interaction to suppress modality-specific bias. Enc-CFA TSDA-Track applies contrastive alignment to encoder representations after interaction to strengthen target-level cross-modal correspondence. Both variants retain a shared inference pipeline without modality-specific branches. Experiments on LasHeR, and zero-shot evaluations on RGBT234 and GTOT under multiple cross-modal protocols demonstrate improvements over representative state-of-the-art trackers. For instance, under the modality-switch protocol on RGBT234, Pre-AFA TSDA-Track achieves an SR/PR of 43.2/56.0, compared with 36.8/50.0 for ToMP-101 baseline. In addition, a study on Anti-UAV-024 further verifies the applicability of TSDA-Track to aerial tracking. Our study highlights the effectiveness of feature alignment domain adaptation for cross-modal tracking.

Tue 29 SeptComputer Vision and Pattern RecognitionArtificial Intelligence
The gist
Tracking objects in videos usually assumes both the initial picture of the object and the later frames come from the same type of camera. The authors tackle a harder problem where these images come from different types of sensors, like visible light and infrared, which makes tracking harder. They propose a method that trains a model to better align features from different sensors so the system can follow the object across different types of images. Their tests show this method improves accuracy in finding the object in cross-sensor situations.
Open → 2609.38637v1

Inspection sparsification improves robot tours with fewer waypoints

Inspection-SPARS: Task-Oriented Sparse Roadmaps for Inspection Planning

Abstract: Inspection planning seeks a minimum-length collision-free robot tour that observes a given set of points of interest (POIs). Sampling-based methods reduce this continuous problem to a graph inspection planning (GIP) problem over a discrete roadmap, which is then solved using combinatorial solvers. Dense roadmaps capture diverse inspection viewpoints and motion shortcuts, and thus admit higher-quality solutions, but they induce large combinatorial search spaces on which state-of-the-art GIP solvers struggle to find good solutions within practical time budgets. Roadmap sparsification---restructuring a dense roadmap into a compact representation that preserves connectivity and path lengths---can alleviate this burden. However, existing sparsification approaches are either agnostic to the underlying inspection task, or strive to ensure coverage of the POIs without accounting for the quality of the resulting inspection plan. We present Inspection-SPARS, which is, to our knowledge, the first inspection-roadmap sparsifier with POI coverage and path-quality guarantees relative to the dense roadmap. To this end, we generalize the SPARS framework, a popular task-agnostic sparsifier, from purely geometric criteria to task-oriented ones, introducing an inspection-aware vertex admission mechanism that treats POI coverage as a first-class sparsification criterion alongside connectivity and path quality. Experiments in realistic 3D environments show that Inspection-SPARS reduces vertex and edge counts by 4-8x while preserving coverage, allowing the GIP solver to compute tours up to 25% shorter than with the dense roadmap or state-of-the-art inspection roadmap. More broadly, Inspection-SPARS shows that sparsification can be made task-aware without sacrificing guarantees on solution quality.

Mon 28 SeptRobotics
The gist
Planning robot paths to check certain points needs lots of possible routes, which slows down finding the best path. The authors present Inspection-SPARS, a method that shrinks the search map while still covering all points and keeping route quality high. This helps robots find shorter paths faster by focusing only on important waypoints. Their tests in 3D environments show it reduces route size by 4 to 8 times and can produce tours up to 25% shorter than before.
Open → 2609.35516v1

GPU-parallel robot motion planner improves timing in dynamic spaces

ST-pRRTC: Parallel Space-Time RRT-C with Adaptive Goal-Time Forests

Abstract: We propose ST-pRRTC, a GPU-parallel space- time RRT-Connect motion planner for problems with known obstacle trajectories and unspecified arrival time. Searching over many arrival times broadens temporal coverage but divides a finite planning budget among more backward trees. To address the challenge, ST-pRRTC builds a shared forward tree and an adaptive forest of backward goal-time trees. Its interval root formulation samples goal arrival times continuously and guarantees probabilistic completeness and asymptotic arrival- time optimality under the stated assumptions in a bounded time domain. The practical root recycling policy has no such guar- antees. It adapts a fixed number of backward trees, replacing later roots while retaining useful search progress. Experiments on three dynamic benchmarks show that both variants achieve lower mean first-solution times and earlier mean final arrivals than ST-RRT* and SI-RRT on problems solved by all compared methods. Further experiments demonstrate the benefit of recy- cling over broad arrival-time ranges. Real-robot demonstrations show root-recycling ST-pRRTC planning motions for a UR5e among moving Crazyflie quadrotors.

Thu 24 SeptRobotics
The gist
Planning robot movements around moving obstacles can be tricky, especially when the exact time to reach the goal isn't fixed. The authors developed a method called ST-pRRTC that uses many quick computations on a graphics card to explore possible arrival times and routes. This method keeps track of multiple possible goal times adaptively, helping it find efficient routes faster than earlier approaches. They tested their method in simulations and real robots, showing it reaches goals sooner while avoiding moving obstacles.
Open → 2609.30533v1

Adaptive communication improves multi-robot search coordination and efficiency

AC-DC: Adaptive Communication for Scalable Dynamic Average Consensus in Multi-Robot Ergodic Search

Abstract: We study scalable peer-to-peer dynamic average consensus (DC) for multi-robot systems under finite-range, finite-rate, and interference-constrained communication. We introduce Adaptive Communication for Dynamic Average Consensus (AC-DC), which jointly adapts Who communicates with whom, When, and over What parts of the consensus state, using local inputs and successfully received neighbor information. Each robot's consensus state estimates the current average of the robots' local inputs. AC-DC updates these estimates as local inputs change and averages the values exchanged between robot pairs. In AC-DC, robot pairs update without waiting for every robot to complete a communication round, and the selected-state messages carry consensus state coordinates independent of team size for a fixed state representation. We apply AC-DC to dynamic-priority multi-robot ergodic search: one consensus stream estimates team visitation for motion coordination, while the other fuses regional measurement information to update uncertainty maps and search targets. Across twelve settings with up to 80 robots and 20 paired trials per setting, AC-DC has the lowest mean (i) normalized covariance-trace area under the curve (AUC) and (ii) attempted modeled communication payload among the compared decentralized methods. Averaged across settings, AC-DC achieves paired AUC reductions of 27.5% relative to state-of-the-art baselines, with 8.7x less communication traffic. As the number of robots increases, we observe that AC-DC's communication payload approaches that of the ideal centralized baseline (one ground compute-station communicating directly with all robots): with 120 robots in a fixed 600 x 600 m scaling test, AC-DC uses 19.3 MB versus 19.2 MB for the ideal centralized baseline, while remaining peer-to-peer.

Mon 21 SeptRobotics
The gist
Coordinating many robots to work together can be hard because they need to share information quickly, but wireless communication has limits like range and data capacity. The authors created a method called AC-DC that helps robots decide who talks to whom, when, and what pieces of information they share, adapting as things change. This way, robots can better estimate group data without waiting for everyone at once, which saves communication and speeds up teamwork. Their tests with up to 120 robots showed better search results with much less communication traffic compared to other methods.
Open → 2609.24702v1

Context-aware fire image augmentation improves multiclass segmentation accuracy

Multiclass Semantic Segmentation of Wildland Fire Images Using Context-Aware Centralized Copy-Paste Data Augmentation

Abstract: Producing accurate annotations for deep learning based image segmentation is both costly and labor intensive. This challenge is especially evident in wildland fire applications, where accurately labeled datasets are scarce due to the difficulty of collecting and annotating dynamic fire scenes. To address this problem, our previous work introduced the Centralized Copy-Paste Data Augmentation (CCPDA) method for semantic segmentation of wildland fire imagery, which generates artificial training samples by randomly pasting fire clusters from source images onto target images. However, random placement can produce contextually unrealistic scenes, such as fire burning on asphalt. In this paper, we present a context-aware strategy designed specifically to improve data quality and realism in small multiclass wildland fire datasets, ensuring that augmented samples remain contextually meaningful. The proposed method restricts fire placement to semantically valid target regions and selects the location whose Ash-Vegetation composition most closely matches the source context. This approach preserves existing fire regions in the target image, prevents unrealistic placements, and maintains contextual accuracy by generating images that resemble real wildland fire scenes. We evaluate the Context-Aware CCPDA strategy through numerical analysis and comparisons with other augmentation methods by a weighted sum-based multi-objective optimization (MOO) approach. The results confirm that the context-aware data augmentation strategy leads to improved segmentation performance and contextual realism, outperforming other augmentation procedures.

Fri 18 SeptComputer Vision and Pattern RecognitionMachine Learning
The gist
Accurately labeling images of wildfires for computer training is hard and expensive, especially because fire scenes change quickly. The authors previously created a method that copies fire parts from one image and pastes them onto another to make more training pictures. Now, they improved this method to only paste fire in places that make sense, like vegetation areas, not on roads. This smarter pasting helps computers learn better where fire is in pictures, making the fire detection more accurate.
Open → 2609.21241v1

Learned occupancy predictions improve robot mapping and navigation decisions

Rethinking Learned Occupancy in Autonomous Active Mapping with Observation-Gated Filtering

Abstract: Autonomous 3D active mapping requires a space robot to choose where to sense while building the geometry needed for navigation. Learned occupancy completion extends spatial context beyond the current field of view, but one predicted map often serves two planning roles: it scores expected surface gain and constrains collision-free motion. Unsupported occupancy can therefore distort both where the robot looks and where it believes it can travel. We study this coupled interface in a controlled closed-loop benchmark by holding the active-mapping system fixed and varying only its planner-facing occupancy across observation-only, learned, oracle-corrected, and ground-truth conditions. Improving occupancy accuracy does not monotonically improve closed-loop coverage: across 25 starts, planning with ground-truth occupancy reaches 70% of the learned baseline's final coverage 12.7 steps earlier on average, while increasing final coverage by only 0.031. Guided by this diagnosis, we introduce an observation-gated filter that retains completion in insufficiently observed regions and suppresses predictions only after repeated frustum exposure without nearby RGB-D support. The filter improves both targeted failure-prone starts without retraining or ground truth. These results motivate online revision of planner-facing geometry during autonomous intervals between communication windows. The current study assumes benchmark RGB-D observations and sufficiently accurate pose estimates; planetary sensing conditions and accumulated localization drift remain to be evaluated.

Tue 8 SeptRoboticsComputer Vision and Pattern Recognition
The gist
Robots mapping 3D environments must decide where to look next and how to move safely. Using learned predictions helps fill in parts of the map the robot hasn't seen yet, but relying too much on guesses can cause mistakes in both planning where to sense and where to travel. The authors tested different ways to provide the robot with occupancy maps and found that better map accuracy doesn't always mean better exploration. They created a filter that uses actual observations to decide when to trust predictions, improving mapping without extra training.
Open → 2609.09069v1