Google’s AI summaries for health often cite lower quality sources
Does the Citation Matter? Quality, Grounding, and Physician-Rubric Accuracy of Google's AI Overviews for Medical Information
Information RetrievalComputation and LanguageComputers and SocietyHuman-Computer Interaction
Summary
Google’s AI-generated health summaries appear in most medical searches, but they often rely on different and lower-quality websites than traditional search results. The authors found that many AI statements either lack proper citations or are not supported by the sources they do cite. When evaluated by doctors, these AI summaries scored low on accuracy and completeness, sometimes including harmful information. Surprisingly, better sourcing did not always mean a more accurate summary.
What this means in practice
- •For healthcare software developers: Design systems that flag or improve AI health summaries by comparing their cited sources against established medical standards.
- •For digital health content teams: Adjust editorial practices to better align AI-generated health content citations with trusted academic and government sources.
Authors
Sam Agnoli, Brigid Reilly, Lily Barna, Emőke-Ágnes Horvát
Abstract
Google's AI Overviews (AIOs) are the first result users encounter when searching the web, shifting the work of selecting and evaluating sources from the user to Gemini. This may be especially consequential for health information, given that Google's traditional algorithm was designed to prioritize quality sources, and inaccurate information carries health risks. We evaluated 3,611 benchmark health questions, comparing each AIO against that question's top five traditional results on source selection and quality. We also evaluated the AIOs' statement grounding in their cited sources and how accurate AIOs were against physician-written rubrics. Overviews appeared for 96.4% of questions but drew on substantially different evidence than traditional search: only 44.4% of the top five results were cited by the corresponding AIO, which instead overrepresented commercial health publishers (e.g., WebMD, Healthline) at the expense of academic and government sources (e.g., Cleveland Clinic, National Health Service). AIO-cited pages scored lower than traditional results on four of six well-established credibility criteria. The grounding of AIO statements to their sources was poor: after correcting for classifier error, an estimated 56.7% of cited statements were supported by their cited page, and a further 20.8% of all statements were factual claims with no citation at all. When AIOs were graded against physicians' rubrics, they earned 38.2% of all available points (e.g., accuracy 54.2%, completeness 21.6%), and among the subset of AIOs whose physician rubric anticipated a specific inaccurate or potentially harmful claim, 22.2% of AIOs contained the harmful content. Notably, the accuracy of AIOs was unrelated to the grounding of their statements. We recommend AIO literacy interventions for users and holding AIOs at least to Google's own health-ranking standards.