language learning platform teams

Abstract: Cantonese lyric writing requires close alignment between lexical tones and melodic pitch. Existing melody-guided lyric generation methods typically rely on symbolic melody to generate lyrics. However, in real songwriting scenarios, melodies are often expressed as raw singing audio or hummed recordings, where pitch is implicit, noisy, and unstructured, making these methods difficult to apply directly. To address this limitation, we propose ARIA, a two-stage audio-driven melody-tone relation modeling framework for Cantonese lyric authoring that generates Cantonese lyrics from singing recordings with provided character-level timestamps. Specifically, we first design a Tri-Stream Relation-Aware Tone Estimator (TRATE) to predict 0243 sequences from timestamped singing audio by modeling multi-stream acoustic cues and relational tonal structure. We then propose a Decoupled Retrieval-Augmented Tone-Conditioned Lyric Generator (DRA-TCLG) to generate fluent lyrics conditioned on predicted tonal plans with retrieval-enhanced lexical guidance. Moreover, we construct a large-scale aligned audio-Jyutping-0243 dataset from real Cantonese singing recordings to support this new task. Experimental results demonstrate that ARIA achieves strong performance in both 0243 prediction and tone-consistent lyric generation, validating the effectiveness of the proposed framework.

Abstract: The widespread availability of LLMs is challenging written learning activities, as students can increasingly generate responses without necessarily engaging with the learning content. Conversational AI creates an opportunity to redesign these activities as dialogue, while multilingual capabilities may make such dialogue more accessible to students learning through a non-native language. We conducted a field study with 305 native Kannada-speaking undergraduate students at English-medium institutions in Karnataka, India. We compared written response activities with text- and voice-based dialogic activities with AI, each conducted in English-only or bilingual Kannada-English settings. Students who completed dialogic activities spent more time on the activities, contributed more, and reported greater interest and self-efficacy than those completing written responses, although fewer students completed the dialogic activities overall. Knowledge increased across all conditions, with no reliable differences in gains between activity formats or languages. Language shaped participation differently across modalities: bilingual interaction was particularly beneficial in voice dialogue, where it reduced articulation difficulties, increased turns demonstrating understanding, and reduced conversation abandonment. However, students also valued English because of its connection to their academic and professional aspirations. These findings show that designing dialogic learning with AI requires more than choosing between writing and dialogue, voice and text, or English and students' native languages. We highlight opportunities to give learners greater control over modality, information, and pace, and to use native languages as translanguaging support rather than as a replacement for English.