Synthetic data improves real object detection in safety monitoring
From Unity Simulation to Diffusion-Based Augmentation: Quantifying Dataset Balance for Robust Object Detection
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Collecting real images for training computer vision models can be hard and risky in places like construction sites. This paper studies how using computer-made images from two methods—video game-like simulation and AI-based image generation—helps make object detection better when there isn’t much real data. The researchers found that mixing mostly real images with a small amount of synthetic ones improves accuracy, while using only synthetic images lowers it. The best results came from adding a bit of both types of synthetic images to real data.
What this means in practice
- •For construction safety teams: Enhance object detection models with combined synthetic and real images to improve safety monitoring accuracy in environments where real data is scarce or risky to collect.
- •For autonomous vehicle developers: Improve training datasets for vehicle object detectors by mixing limited real-world data with simulated and AI-generated images to reduce data collection costs and improve model robustness.
Authors
Mohamed Benkedadra, Aissa Saoudi, Maxime Gloesener, Sidi Ahmed Mahmoudi, Matei Mancas
Abstract
Modern computer vision models achieve high accuracy when trained on large-scale annotated datasets. In critical domains such as construction safety monitoring, data collection is costly, hazardous, and ethically constrained. This paper presents a systematic study comparing two complementary data generation paradigms, (1) Unity Simulation-based rendering and (2) Controllable Diffusion-based generation (CIA), for object detection under real data-scarce conditions. A unified experimental framework enables controlled dataset mixing across real, simulated, and generative sources, while maintaining identical model and training settings. Quantitative evaluation using Precision, Recall, mAP, and custom $Δ$-metrics, reveals that neither simulation nor generative augmentation alone achieves optimal transferability. Unity-only training yields an mAP@0.5 drop of $-50\%$ relative to real data, while CIA-only training shows a milder $-16.5\%$ degradation. Hybrid compositions significantly improve performance, with the 90\% real + 10\% Unity configuration achieving the best overall mAP@0.5 of $62.68\%$ ($+7.64\%$ over baseline), and the 90\% real + 10\% CIA configuration maximizing precision at $74.45\%$. Results demonstrate that limited synthetic inclusion enhances generalization, while excessive substitution induces domain drift.