Cross-Sign Language Transfer Learning Using Domain Adaptation with Multi-scale Temporal Alignment
2026-08-17 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors studied ways to recognize sign languages using computer programs. They found that a method called Domain Adaptation, which helps the program learn from one sign language to understand another better, works better than just simple transfer learning. They also discovered that looking at short-term movements in the signs helps improve recognition, especially for American Sign Language. Finally, they tested two types of video input, RGB and Optical Flow, and found that RGB videos generally gave better results.
Sign Language RecognitionTransfer LearningDomain AdaptationTemporal Relational Network (TRN)Multi-scale Temporal RelationsAmerican Sign Language (ASL)RGB VideoOptical FlowNeural Networks
Authors
Keren Artiaga, Yang Li, Ercan Engin Kuruoglu, Wai Kin, Chan
Abstract
Sign language serves as a vital means of communication for individuals with hearing impairments, yet recognition resources for the over 100 distinct sign languages are severely lacking. In response, we present our work on sign language recognition using transfer learning and the domain adaptation method TA3N, which utilizes the Temporal Relational Network (TRN) module for aligning multi-scale temporal relations. Our findings highlight the superior performance of Domain Adaptation to neural network-based transfer learning, particularly in improving recognition of American Sign Language (ASL). Our research also identifies the effectiveness of aligning shorter-term temporal features between source and target domains. In addition to using RGB, we conducted experiments using Optical Flow mode for the sign language samples, ultimately determining that RGB outperforms Optical Flow in the majority of cases. Our work aims to improve accessibility and communication for individuals who rely on sign language as their primary mode of communication.