SICAGE: Speaker-Independent Culture-Aware Gesture Generation using TED4C-L Dataset
2026-06-29 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionGraphicsHuman-Computer InteractionSound
AI summaryⓘ
The authors created SICAGE, a system that helps computers generate hand and body movements during speech that match different cultures without confusing who the speaker is. They teach the system to recognize cultural styles using sounds and words, making sure it focuses on culture instead of individual speaking habits. They tested their approach on a big collection of TED talks from speakers of four cultures and found it made gestures look more natural, diverse, and culturally appropriate. They also built ALaDiT, a fast tool that uses these cultural insights to create gestures in real-time.
Co-speech gesture generationCultural embeddingsDomain generalizationAdversarial learningFishr regularizationMultimodal generatorDiffusion modelsSpeaker-disjoint splitsGesture diversityTED talks dataset
Authors
Ariel Gjaci, Antonio Sgorbissa, Vittorio Murino
Abstract
Recent co-speech gesture generation methods often overlook cultural differences, limiting their effectiveness in human-agent interaction. Moreover, culture-conditioned models are rarely evaluated under speaker-disjoint splits, so apparent "cultural" behavior may be confounded with speaker-specific gesturing style. We introduce SICAGE, a modular framework for culture-aware co-speech gesture generation that conditions motion synthesis models on speaker-independent cultural representations. SICAGE learns these representations from audio and text by treating each speaker as a separate domain while imposing invariance across speakers. This encourages representations to remain culture-discriminative while reducing dependence on speaker identity. The resulting cultural embeddings condition a multimodal generator to produce culturally appropriate gestures. We instantiate this idea with two domain generalization approaches: adversarial learning and Fishr regularization. We further introduce ALaDiT, a real-time diffusion-based gesture generator designed to efficiently incorporate the learned cultural embeddings. To validate our method, we built TED4C-L, a 106-hour multimodal dataset of 764 TED speakers from four cultural groups. Experiments show that SICAGE improves motion realism, diversity, beat synchronization, semantic relevance, and cultural consistency.