Benchmark dataset evaluates AI responses to indirect self-harm expressions
VEILED-SH: Vulnerability Expression through Indirect Language Evaluation Dataset (Self-Harm)
Computation and LanguageArtificial IntelligenceComputers and Society
Summary
AI systems sometimes fail to recognize when people express distress or self-harm indirectly, especially when cultural differences matter. The authors created VEILED-SH, a large collection of examples from Singapore and public sources, to test AI responses in different settings like counseling chats and forums. They developed a detailed system to label the intentions and suitable responses, and compared how different AI models align with expert human profiles. Their study showed that AI systems vary in how well they handle indirect and culturally nuanced expressions of self-harm.
What this means in practice
- •For mental health platform developers: Test AI tools against culturally nuanced expressions of distress using VEILED-SH for safer user interactions.
- •For social media moderation teams: Improve detection of subtle self-harm or suicide-related content by evaluating automated responses with the benchmark dataset.
Authors
Leanne Tan, Keith Chua, Shaun Khoo, Onno P. Kampman, Lim Ruling Charmaine, Hazirah Hoosainsah, Kellie Yu Hui Sim, Kai He, Kenny Tsu Wei Choo, Roy Ka-Wei Lee
Abstract
AI systems may miss indirect, culturally situated distress or provide support mismatched to the application context. Existing benchmarks do not jointly address cultural variation, application context, and practitioner disagreement. We introduce VEILED-SH, a benchmark and construction methodology for evaluating single-turn responses to indirect expressions of suicide, self-harm, and distress. Its 10,003 prompts span counselor chat, forum posts, platform search, and triage, drawing on Singapore-grounded sources and public datasets. An expert-validated multi-label taxonomy distinguishes intent, communicative function, and response behavior. Six automated profiles, individually calibrated to three peer-support practitioners and three clinicians, retain separate reference policies for proxy-alignment scoring. Evaluating five systems on a coverage-enriched sample reveals variation in proxy alignment across profile-generated reference policies, even for the same system responses.