Sign Language Recognition Using Original and Synthetic Depth Image Based Point Cloud Data Models
2026-08-10 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors explored using depth images to improve sign language recognition, a field mostly focused on RGB images due to limited depth image datasets. They created synthetic depth images from regular RGB images using a special neural network called Depth Anything V2. They then compared how well different neural network models recognized sign language from point clouds made from both real and synthetic depth images. Their results showed that models using real depth images generally worked better, but in some cases, synthetic depth images performed just as well or better. This suggests synthetic depth can be useful when real depth data is not available.
Sign Language RecognitionRGB ImagesDepth ImagesPoint CloudsPointNetDepth Anything V2Synthetic DepthNeural NetworksLong Short Term Memory (LSTM)Gesture Recognition
Authors
Rustem Ozakar, Eyup Gedikli
Abstract
Research regarding the sign language recognition mostly relies on RGB images, whileas sign language datasets that provide depth images are limited. Point clouds obtained from depth images can be used for sign language recognition with neural networks like PointNet. In recent years, various neural networks are used for generating realistic depth images from monocular RGB images. In this work, synthetic depth images were created from RGB images using Depth Anything V2 network. For this purpose, three sign language datasets (Real-time ASL Fingerspelling, KArSL, AUTSL) which contain both RGB and depth images were used. Classification accuracies of the point cloud data created from both original and synthetic depth images using various PointNet architectures were measured for sign language recognition. From the original and synthetic point clouds, frame based, Point Gesture Map and Long Short Term Memory data models were used for classification and their performances were compared. In the results, both original and synthetic based data achieved acceptable performance in most models. In general, original depth based point cloud models performed better than synthetic ones, however in some models synthetic depth based models performed better than the originals.