Papers for

robotic perception teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Mixed signal design uses capacitor memory for event camera segmentation

EventSpikeSeg: Memory-Aware Mixed-Signal Architecture for Event-Camera Spiking Segmentation

Abstract: Event-camera segmentation requires persistent neural state despite sparse input events, making memory organization a central constraint on embedded implementation. We present EventSpikeSeg, a mixed-signal architecture combining resident capacitor memory with shared analog processing in GF180MCU 180-nm CMOS. Its 25-population spiking encoder-decoder has 143,824 synaptic weights but requires 30,976,000 membrane states at 440 x 640 resolution. The mapping specifies chronological 5-ms model updates, digital event/address services and 64-state capacitor banks. Characterization covers 22 module families and 74 operating points, distinguishing transistor simulations from digital-library estimates. Nominal results include 41.711 microwatts biased neuron-core power, 3.338-pJ positive terminal energy for a 1.4-microsecond cell read, and 0.531-mV selected-state drift over a 5-ms extracted-tile hold. Commit and coupling errors are reported separately. The software checkpoint records 27.976% mean intersection-over-union on a held-out six-class DSEC split; this is not hardware accuracy. Component layouts and simulation evidence quantify local implementation costs and expose storage replication and service scheduling as scaling constraints. A detailed SPICE-calibrated behavioral forward simulation of one full-resolution DSEC frame achieves 24.26% mean intersection-over-union, versus 26.22% for the matched PyTorch reference, using a compact 5-fF bank variant and external count/voltage calibration. Physical full-network operation, system energy and mismatch robustness remain unverified.

Fri 13 NovHardware ArchitectureComputer Vision and Pattern RecognitionNeural and Evolutionary Computing
The gist
Event cameras capture changes in a scene as events, which are sparse and require memory to interpret over time. The authors introduce EventSpikeSeg, a new hardware design that uses a mix of analog and digital circuits with capacitor memory to process these event streams more efficiently. Their design is tested through simulations that estimate power use and accuracy, showing promise but not yet demonstrating a fully working physical system. This work focuses on how to organize memory and computation for event camera segmentation in embedded hardware.
Open → 2611.18524v1

Color affects how humans and AI recognize partly hidden objects

Impact of color in computer and human-based object recognition under occlusion

Abstract: Partial occlusions challenge object recognition in both human and computer vision (CV), but the impact is largely dependent on the occlusion and model design. Properties relating to color are particularly important in biological recognition and frequently leveraged for camouflage, however there are still open questions about what makes them effective and how they translate to CV. This work studies partial occlusions that create camouflage, including those that blend in with target objects or with scenes, which alter object representations in distinct ways. Experiments evaluate recognition performance on both human participants and models ranging from convolutional neural networks (CNNs) to transformers, and additional analysis is performed on models to uncover trends in processing and object representations under multiple occlusion types. Novel metrics offer explainability of model behavior under occlusion: non-occlusion precision (NOP) and non-occlusion recall (NOR) measure alignment of model activations with and without occlusion, and activation in occlusion (AIO) measures the percentage of activations that occur in occluded regions. Findings show that human recognition, and most transformers, are negatively impacted by generated camouflaging occluders, with scene-based camouflage performing worse than object-based. Classical CNNs show a strong opposite trend, performing the worst on neutral-colored occlusions, suggesting that contextually-related colors, particularly scene-based, benefit object recognition rather than disrupting shape or boundary cues. Additionally, generated camouflaging occluders for training augmentation improves robustness to unseen camouflage for selected CNN model, and occlusion metrics suggest trends about learned mechanisms. Findings have interesting implications for alignment in human and computer vision, model vulnerabilities, and explainability.

Fri 6 NovComputer Vision and Pattern Recognition
The gist
Recognizing objects when they are partly covered is hard for both people and computers. This paper looks at how color influences this challenge, especially when the covering blends in like camouflage. The authors compare human recognition with different types of AI models and find that color camouflage affects them differently. For example, human and transformer models struggle more with camouflage that blends into the scene, while some CNNs do better when color matches the background. Training AI with camouflaged objects can help some models recognize hidden objects better.
Open → 2611.08808v1

Closed-form triangle splatting speeds novel-view synthesis with mesh priors

CF-Splat: Closed-Form Triangle Splatting with Mesh Priors

Abstract: Splatting-based radiance-field representations have recently gained popularity in robotic perception, enabling high-quality novel-view synthesis (NVS) through explicit scene primitives. However, existing splatting methods typically rely on iterative nonlinear optimization that jointly optimizes millions of coupled primitive parameters, resulting in substantial computation time, peak GPU memory consumption, and susceptibility to local minima. In this paper, we propose CF-Splat, a novel closed-form triangle-splatting framework for efficient scene appearance estimation over a known mesh either pre-existing or provided by upstream mapping or reconstruction pipelines. Our key insight is that, by properly formulating the problem over the given mesh geometry and calibrated cameras, the rendered pixel colors can be expressed linearly with respect to the per-vertex spherical harmonics (SH) coefficients, allowing multi-view scene appearance estimation to be formulated as a global linear least-squares (LLS) problem without the need of iterative nonlinear optimization. To guarantee a well-posed solution under uneven multi-view observation coverage, we introduce adaptive SH regularization that constrains insufficiently supported high-order SH coefficients and yields a unique global minimizer. To ensure the scalability to large-scale scenes, we develop a triangle-structured matrix-free preconditioned conjugate-gradient solver that avoids explicitly constructing the large sparse normal system while producing the same LLS solution. Experiments on diverse real-world datasets demonstrate that CF-Splat achieves competitive or superior NVS quality compared with state-of-the-art splatting methods. While, CF-Splat requires approximately 27.5-378.0 times less computation time and 1.8 times lower peak GPU memory on average than 3DGS across scenes of different scales on a consumer-grade laptop.

Thu 15 OctRobotics
The gist
Rendering images of a 3D scene from new viewpoints usually takes a lot of computation and memory. The authors present CF-Splat, a method that uses a known 3D mesh and camera information to quickly calculate scene colors without slow iterative steps. Their approach solves the problem using math that finds the best fit in one go, making it much faster and more memory efficient than previous methods. This makes creating new views of scenes more practical on common laptops.
Open → 2610.22108v1

Vision language models assessed for true visual causal reasoning skills

CCRV-Bench: Constraint-Based Evaluation of Causal Reasoning in Vision-Language Models

Abstract: Vision-language models (VLMs) have demonstrated excellent performance in visual tasks, but their visual causal reasoning capabilities still lack reliable evaluation. Existing evaluations struggle to distinguish whether a model is performing causal reasoning based on visual evidence or relying on statistical correlations for shortcut learning, thereby potentially overestimating their actual capabilities. This paper proposes CCRV-Bench, a constraint-driven visual causal reasoning benchmark for single-image physical scenarios. We construct an orthogonal framework that evaluates four causal task dimensions: causal relation discovery, state prediction, causal diagnosis, and intervention. We further introduce entity symbolization, spatial grounding, the factual adversarial constraint, and minimalist output constraints to reduce shortcut cues while preserving the physical commonsense required by the task. Experiments across 15 multimodal models show that constraint sensitivity is task- and model-dependent: intervention has the largest average effective degradation among the four causal tasks, spatial grounding is the most damaging constraint on average, and the factual adversarial constraint improves DCR for all evaluated models. These results show that unconstrained performance does not determine constrained robustness and that a single aggregate score can obscure distinct failures in causal identification, spatial grounding, and constraint-compliant expression. CCRV-Bench provides a standardized framework for diagnosing image-grounded causal reasoning under controlled constraints. The code is available at https://github.com/0815linyuan/CCRV-Bench-Constraint-Based-Evaluation-of-Causal-Reasoning-in-Vision-Language-Models

Fri 25 SeptComputer Vision and Pattern Recognition
The gist
Many AI systems that combine seeing and understanding language appear to do well on tasks, but it’s unclear if they really understand cause and effect from images or just guess based on patterns they’ve seen before. The authors created a special test called CCRV-Bench to better check if these models truly reason about causes in single images by controlling tricky clues that can mislead AI. They tested 15 models and found that some tasks and constraints reveal weaknesses that overall scores hide, showing that good scores don’t always mean real understanding. Their benchmark helps pinpoint exactly where these models succeed or fail in causal thinking grounded in what they see.
Open → 2609.30979v1

Multimodal fusion improves robotic new view synthesis from camera and LiDAR

M3GD: Multi-Modal Multi-View Geometric Diffusion for Camera--LiDAR Novel View Synthesis

Abstract: Robotic novel view synthesis (NVS) must recover both visual appearance and metric 3D structure, yet most generative NVS methods rely only on images, overlooking LiDAR, a complementary sensor common on robotic platforms. We present M3GD, a Camera--LiDAR multimodal representation for generative NVS that composes independently pretrained 2D image and 3D point-cloud foundation models without separately pretraining a cross-modal translator. We show that, after camera projection, frozen LiDAR and image features exhibit substantial shared spatial structure, providing a natural cross-modal representation. M3GD conditions generation on LiDAR through this structure: it combines explicit geometry statistics with learned point-cloud descriptors into view-aligned packets on the image-latent grid, injected through a lightweight residual adapter into a multi-view flow-matching generator whose latent space, decoders, and training objective remain intact. On the GrandTour dataset, M3GD improves target-view RGB and depth synthesis over an image-only version of the same backbone. Ablations show that the gains come from pixel-aligned LiDAR content and that target-view LiDAR acts as a geometric query linking the requested view to source observations. Deployment on a ground robot demonstrates practical real-world operation, with a configurable quality--cost trade-off controlled by the number of Euler integration steps.

Thu 24 SeptRoboticsComputer Vision and Pattern Recognition
The gist
Generating new images of a scene from different viewpoints is a key problem for robots, but existing methods typically use only camera images. The authors show that combining camera images and LiDAR data, which measures 3D points directly, improves the quality of new view images and depth maps. Their method, called M3GD, uses pretrained models for images and point clouds and cleverly aligns their features without extra training. This improves the robot’s understanding of 3D space and lets it better synthesize novel views in real-world settings.
Open → 2609.30056v1