Large language models assist with incomplete bioassay metadata annotation
Can LLMs Reliably Annotate Bioassay Metadata to Improve Data Readiness?
Computation and LanguageArtificial IntelligenceDatabases
Summary
Many bioassay records lack important descriptive labels, which makes it hard to use this data for AI research. The authors found that large language models (LLMs) can predict missing labels from text descriptions with high accuracy. These models also helped experts identify errors in existing annotations. While LLMs show promise for automating bioassay metadata curation, careful human review is still needed to ensure quality.
What this means in practice
- •For pharmaceutical data teams: Improve assay database quality by automatically filling and auditing missing metadata labels using LLMs.
- •For chemical informatics engineers: Integrate LLM-based metadata prediction to support better data preprocessing before machine learning model training.
Authors
Laura van Weesep, Riccardo Tedoldi, Jens Sjölund, Hossein Azizpour, Susanne Winiwarter, Ola Engkvist, Jon Paul Janet, Samuel Genheden, Juan Viguera Diez
Abstract
The emergence of foundation models for molecular property prediction requires a high degree of AI data readiness, including reliable metadata annotation. However, both public repositories and industrial screening databases suffer from missing, inconsistent, or conflated assay annotations. In this work, we quantify the extent of missing annotations in PubChem for the BioAssay Ontology (BAO) assay format and physical detection method fields and investigate whether open-source and proprietary large language models (LLMs) can reliably predict and audit metadata annotations directly from the assay text. In our assessment, we found that the annotation coverage across PubChem's $\sim$2 million bioassays is critically sparse, 36\% lacking an assay format, 89\% a BioAssay type, and >99.9\% any BAO-mapped assay format or detection technology term. This motivates the need for automated test-metadata curation. Using evaluation sets derived from PubChem and ChEMBL, we assess the agreement of seven open-source and proprietary LLMs with existing silver labels. Recall is at least 0.96 for biochemical and cell-based assay formats, with a similar pattern for detection technology, although disagreements increase on under-represented classes. Manual inspection shows that many of these disagreements trace back to inconsistencies between silver sources rather than to LLM error. Moreover, in a qualitative study with a senior industrial curator, LLM-generated evidence prompted the expert to revise some of their own labels, showing LLMs can flag potentially mislabeled assays. Across the study, performance differences between proprietary and open-source models were small. Together, these results suggest LLMs can support the large-scale annotation and auditing of assay metadata, though per-class reliability estimates and targeted human review remain necessary before such labels enter downstream ML pipelines.