Multi-view method improves physical consistency in low-light TeX generation
Multi-View Structural Representations for Physically-Consistent TeX Generation
Computer Vision and Pattern RecognitionMultimedia
Summary
Generating images in low-light conditions is hard because visible and infrared pictures show different details. The authors propose a new method called MVR-MoE that combines clues from both types of images more effectively by organizing the information from different views. Their system also makes sure that the generated temperature and texture maps follow known physical laws. Tests show this method works better and is efficient compared to others.
What this means in practice
- •For software engineers: Improve image generation pipelines for low-light scenes by integrating infrared and visible inputs with physics-based modeling for more accurate outputs.
- •For thermal imaging device developers: Enhance thermal image outputs by combining multi-view structural data to produce physically consistent temperature and texture maps in challenging environments.
Authors
Daoheng Li, Jianjun Qian*, Mengkai Yan, Jian Yang
Abstract
Infrared and visible images provide complementary cues for low-light image generation, and physics-based methods can better exploit this complementarity. However, existing physics-based methods still rely heavily on paired data. To address these limitations, we propose Multi-View Routing Mixture-of-Experts (MVR-MoE), a framework for physically consistent TeX generation that learns multi-view structural representations. MVR-MoE mines intrinsic structural cues from infrared and visible images to organize them into multi-view representations, while the proposed multi-view hyper connections expand the feature space and preserve both shared evidence and modality-specific details. A hierarchical routing mechanism further regulates feature utilization at both the structural-view level and the expert level, enabling adaptive exploitation of heterogeneous cues. The predicted temperature, emissivity, and texture maps are constrained by the Stefan-Boltzmann law to ensure physically consistent. Extensive experiments on challenging datasets demonstrate that MVR-MoE achieves superior robustness and discriminability while remaining lightweight.