Papers for

social robotics developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Person actions help predict group intent in vision language models

Do the Details Help the Whole? One-Way Transfer from Person Actions to Group Intent in VLMs on JRDB-Social

Abstract: Social scene datasets label people at two levels: what each person is doing and what the group is there for. Training on both levels at once is often assumed to help each of them. We test this assumption for vision-language models (VLMs) on JRDB-Social, treating person actions as the details and group intent as the whole. We fine-tune Qwen3.5 4B under eight training arms with three seeds, repeat the core arms on Qwen3.5 9B and Qwen3-VL 4B, and judge every arm with a verdict rule fixed before testing. Writing each member's actions before the intent raises intent macro-F1 from 30.1 to 31.0, and writing the intent first reaches 31.2, while action F1 stays within 0.2 points of an action-only model. Under the selected recipe, no arm trained with intent improves actions. The gain requires correct, image-grounded actions: empty action lists or actions borrowed from another group with the same intent lower intent F1. Forced-prefix tests, occlusion, member masking, linear probes and component grafting show that the intent follows the written actions, relies more on people and partly resides in the vision tower. Supplying actions as input instead of as targets hurts both levels.

Thu 12 NovComputer Vision and Pattern RecognitionComputation and Language
The gist
This paper looks at whether knowing what each person in a group is doing helps a computer model understand what the group as a whole intends to do. The authors focus on vision-language models using a dataset called JRDB-Social. They find that providing details about individual actions can slightly improve the model's ability to guess the group's intent, but knowing the group's intent does not help recognize the actions. Correct and specific details about people’s actions are important for this improvement.
Open → 2611.16581v1

Composing audio and text to create synchronized whole-body gestures

CFGComposer: Compositional Audio-Text-to-Motion Generation via Copula Score Correction

Abstract: Generating whole-body gestures that are simultaneously rhythmically synchronized with speech audio and semantically aligned with text descriptions is a fundamental challenge in embodied AI and social robotics. We tackle this Audio-Text-to-Motion (AT2M) problem by composing two independently-trained diffusion models at inference time and refine bias in joint inference by modeling the copula score $s_I$. A rigorous theoretical analysis reveals that naive epsilon blending corresponds to a Product-of-Experts formulation that omits the copula score $s_I$: the cross-modal interaction between audio and text that neither marginal model can capture. We propose CFGComposer: a jointly-trained AT2M diffusion model that estimates $s_I$ via a CFG analogy and injects it as an $ε$-space correction. Experiments on BEAT demonstrate best-in-class semantic fidelity (SRGR $0.9916$, FID $27.18$) with GT-level beat alignment (BA $0.1826$ vs. GT $0.1743$), and 76-83% win rate in blind pairwise human evaluation against all prior methods. Physical deployment on a Unitree G1 humanoid confirms direct sim-to-real transfer.

Wed 11 NovSoundArtificial Intelligence
The gist
Making robots or digital characters move their bodies in ways that match both what they are hearing and what they are supposed to be expressing is very hard. The authors create a new approach that combines two separate motion generators—one that listens to audio beats and one that understands text meaning—and corrects their combined output to better capture interactions between sound and meaning. Their approach produces more accurate and natural-looking gestures that align well with speech rhythm and text meaning. They tested it on a standard dataset and even showed it working on a real robot.
Open → 2611.15101v1

Acoustic features predict speaker familiarity and kinship in conversations

Dyship: Leveraging Acoustic Features For Speaker Relationship Classification From Dyadic Conversations

Abstract: We introduce DyShip, a framework for classifying speaker familiarity and kinship from dyadic conversations in the Seamless Interaction corpus. We compare five speech representations: spectrograms, MFCCs, self-supervised WavLM representations, x-vectors, and estimated articulatory movements. Spectral features perform best for familiarity classification, reaching an accuracy of 87.81%, while WavLM achieves the highest kinship accuracy of 85.17%. X-vectors and articulatory features yield weaker results on familiarity classifiaction, but shine for kinship classification, with an AUC of 0.909 for x-vectors. Most classifiers exceed previously reported human and audio language model scores on the same dataset. These findings highlight the potential of conventional acoustic features for relationship inference and motivate further investigation of the conversational cues captured by different representations.

Tue 10 NovComputation and LanguageArtificial Intelligence
The gist
The paper studies how different types of speech sounds help identify if two people talking know each other well or are related. The authors test five ways to represent speech, like spectrograms and specialized speech feature vectors, on a dataset of two-person conversations. They find some features work better for recognizing familiarity, while others are better for kinship. The results often beat previous human and machine attempts on the same dataset. This shows everyday speech sounds contain useful clues about relationships between speakers.
Open → 2611.14241v1

Pose refined model improves zero shot recognition of social bodily behaviors

PRIMA: Pose Refined Inductive Motion Adapter for Zero Shot Social Bodily Behavior Recognition

Abstract: Recognizing fine-grained bodily behaviors, such as self-grooming, postural shifts, and backchanneling, is a useful intermediate step for computational social signal analysis. But the behaviors that matter vary across social contexts, and retraining a model for each new taxonomy is expensive. Vision-language models (VLMs) can adapt to new categories through text prompts, yet they lack sensitivity to body structure and often confuse subtle postural distinctions. We propose PRIMA (Pose Refined Inductive Motion Adapter), a lightweight module that injects a body-structure prior into frozen VLMs using human pose as a training-time supervisory signal, requiring no pose input at inference. PRIMA separates a model-specific projection head from a shared core that encodes transferable body-configuration knowledge; adapting to a new VLM requires retraining only the head. We train PRIMA on a micro-action dataset and evaluate entirely through zero-shot prompting. The adapter improves recognition on its training domain, transfers to additional VLM backbones at different scales with the core frozen, and generalizes to two unseen target datasets from social psychology and daily activity recognition. In a supporting downstream study, the improved behavior predictions also yield higher correlations with human ratings of rapport and dominance in a collaboration context. Within the scope of the tested models and indoor datasets, the results indicate that pose-guided training can produce body-structure priors that generalize across unseen behavior vocabularies and can be partially reused across VLM backbones while preserving RGB-only inference.

Sun 18 OctComputer Vision and Pattern RecognitionArtificial Intelligence
The gist
Recognizing small body movements like grooming or nodding helps understand social interactions, but these behaviors change depending on the situation. The authors created PRIMA, a lightweight module that guides existing vision-language models using human pose information during training, without needing pose input later. PRIMA helps models better understand body shapes and motions, improving their ability to recognize different social behaviors without retraining for each new category. This works even on new datasets and across different model types, showing better predictions that match human impressions of social signals like rapport.
Open → 2610.25482v1