Deep learning detects multiple video game bugs from gameplay footage
Multi-Label Perceptual Bug Detection in Video Games using Deep Learning on Gameplay Footage
Machine Learning
Summary
Finding bugs in video games by watching how people play is usually slow and expensive because it is done manually. The authors created a deep learning system that can spot several types of visual bugs from short video clips of gameplay at the same time. Their approach uses a model called ResNet-BiLSTM that looks at sequences of frames over time to improve detection accuracy. They also provide a large dataset of gameplay videos labeled with multiple bug types to help train and test their method.
What this means in practice
- •For video game developers: Automatically detect multiple visual bugs from gameplay videos to reduce manual testing effort during game development.
- •For quality assurance teams: Use deep learning models to identify perceptual defects in game visuals across different genres, improving testing coverage and efficiency.
Authors
Nahian Rifaat, Felix Morosov, Loutfouz Zaman
Abstract
Traditional approaches for automated bug detection in video games, such as manual testing, can be beneficial for the improvement of quality assurance, but they can be expensive and time-consuming. The scarce number of tools available to detect multiple perceptual bugs in the same video frame introduces detection challenges for automated bug detection tools in real-world scenarios. We propose a deep learning model for multi-label perceptual bug detection and compare it against video classification models such as Inflated 3D ConvNet and 3D ResNet. Our proposed model, ResNet-BiLSTM, achieved an F1 score of 85.78% on the benchmark dataset. Our results demonstrated that temporal dependency modelling is beneficial for accurate video-based bug detection. We believe this work with multi-label perceptual bug detection on gameplay videos will help save resources spent on manual testing workloads in video games. Furthermore, we introduce a new dataset with multi-label perceptual bugs in this work. The dataset contains 77,969 video clips across different genres of games with approximately 1.2 million frames, containing combinations from 5 classes of bugs in the same video frame.