Thermal cameras enable privacy-safe indoor human detection and monitoring
Towards Privacy-Preserving Thermal Human Perception from Dataset to Deployment
Computer Vision and Pattern Recognition
Summary
Seeing people inside buildings usually requires regular cameras, which can invade privacy. The authors created a big set of paired thermal and normal images from indoors to help detect and understand people without showing clear pictures. They trained and tested computer programs to find people and their activities using only thermal images and checked how well they work in new rooms. They also tried turning thermal images into regular-looking ones and even tested their method in real sleep monitoring devices with nearly 1,000 units, showing it can work in real life while keeping privacy safer.
What this means in practice
- •For smart home system developers: Use the provided indoor thermal dataset and models to improve privacy-respecting occupant detection in home automation systems.
- •For healthcare device manufacturers: Deploy privacy-preserving thermal perception models in sleep-monitoring devices to detect and monitor patients without intrusive cameras.$Commercial implications: This enables production of sleep-monitoring products that respect patient privacy while providing reliable human perception.
Authors
Kailai Sun, Yue Cui, Xinwei Wang, Zijie Gao, Baoshen Guo, Zhiqing Hong, Irfan Qaisar, Xu Yang, Yueting Li, Dian Fan, Qianchuan Zhao
Abstract
Indoor human perception is the foundation for smart homes and healthcare. However, RGB cameras raise privacy concerns in private indoor environments, while low-resolution thermal cameras provide limited spatial and semantic information. Large-scale indoor thermal datasets and real-world deployments remain limited. Here, we introduce Indoor RGB-T, a large-scale paired RGB--thermal dataset for indoor human perception. The dataset covers 20 indoor rooms and contains 90,576 80x62 thermal frames, 87,016 temporally matched RGB--thermal pairs, and 115,557 annotated human bounding boxes. We establish benchmarks for robust thermal occupant-state detection with 0.75 mAP and occupancy estimation with an accuracy of 0.847 in unseen rooms. We further evaluate RGB-to-thermal and thermal-to-RGB translation, achieving a temperature MAE of 0.654 C and an R2 of 0.923. Finally, we deploy the thermal perception model on a commercial sleep-monitoring system in the real world. Field testing with 100 participants and deployment of approximately 1,000 devices further demonstrate its practical feasibility. We identify practical challenges that motivate future research directions, including environmental thermal interference, sensing range and multi-person occlusion. The code, trained models and partial dataset are available at https://github.com/kailaisun/RGBT and https://huggingface.co/skl24/RGBT.