health policy makers

Abstract: People increasingly bring their health questions to ChatGPT, yet safety evaluations of LLM health advice rest almost entirely on clinician-authored vignettes, synthetic prompts, or simulated user profiles. The gap is widest for the Global South, where concerns about demographic bias and unsafe recommendations are well documented but have never been tested on real usage. We present, to our knowledge, the first demographically grounded safety evaluation of real-world health conversations with ChatGPT. From a consent-based data-donation corpus of 202,590 conversations from 1,252 users in India, Nigeria, Brazil, and Pakistan (December 2022 to February 2026), each linked to self-reported age and gender, we identify 15,555 health conversations, classify their health intent using an LLM classifier adapted from an established taxonomy, and score response safety under both a context-blind and a user-context-aware rubric. Roughly one in five health conversations receives a high-risk rating. Risk concentrates in symptom and condition queries and recurs through identifiable failure modes: unsafe medication or dosage guidance, harmful self-diagnosis, missing safety warnings, encouraging over-reliance on the model, and failure to refer users to professionals. Supplying the user's own accumulated context raises the high-risk share by 13-17 percentage points, and about one in four conversations rated safe in isolation turns out to be unsafe for the specific person asking. Safety evaluations built on synthetic cases are likely to miss these harms.

Abstract: Academic collaboration is of self-evident value but requires a quantitative representation to be optimally guided by policy. No established methodological approach to such representation exists. Here we introduce a general framework of graphical and bibliometric analysis of open data for the task of quantifying the impact of academic networks, with NIHR Biomedical Research Centre (BRC) clinical-academic partnerships in England as the prototype. We define publication-level identities for the 20 English BRCs based on the conjunction of authors from each BRC's partner institutions. Drawing on bibliometric and administrative records, we characterise the graphical properties of each network, estimate what the university adds to the hospital's papers, what the partnership adds to the papers of relatively infrastructure-poor collaborating institutions, and how that gain depends on existing infrastructure. We apply our framework to the NIHR UCLH/UCL BRC as an exemplar. UCLH/UCL authored 20,985 network papers from April 2007, in collaboration with 9,868 distinct external partners over the whole record, forming the most central node of the graph of networks across England. University co-authored papers exhibited 1.6 times the field-weighted citation impact (FWCI) of hospital-only papers, and were 2.1 times as likely to be cited by a patent. Across 60 of the exemplar's most partnered with UK healthcare organisations, the benefit rose from 1.8 times where local NIHR infrastructure activity was densest to 3.4 times where it was sparsest, while impact without the exemplar varied little. Academic networks can be robustly identified from open data, enabling comparative analysis of collaborative impact. Applied to NIHR BRCs, the approach enables quantification of the impact across networks and reveals that benefit is most pronounced where infrastructure is least developed.

Abstract: Should multilingual LLMs answer medical questions consistently across input languages, or adapt responses to cultural cues? Existing multilingual medical benchmarks usually assume that medically correct answers should remain consistent across languages and treat cross-lingual variation as model error. In contrast, cultural adaptation research argues that appropriate medical answers may legitimately differ across contexts. We review the multilingual medical NLP literature through these two perspectives, we identify three gaps: limited stakeholder perspectives (e.g., of medical professionals), a lack of empirical evidence on which approach better serves users, and no benchmarks capable of distinguishing universally correct from culture-specific cases. To address the first gap, we survey 356 participants across three stakeholder groups (medical, NLP, and anthropology professionals) in three countries (Germany, Spain, and the United States). Anthropologists consistently favor adaptation, while medical and NLP respondents remain divided, with notable divergence between U.S. and European medical professionals. LLMs prompted with profession and country personas fail to reproduce this variation, overestimating cross-lingual consistency preference among NLP and medical personas. We conclude that neither consistency nor adaptation can currently be considered clearly preferable, highlighting the need for empirical evidence on which approach better serves users across cultural contexts.