Fine-tuned small language model improves protein function annotation
QLoRA Fine-Tuning of Ministral LLM for Sequence-to-Function Protein Annotation
Computation and LanguageArtificial IntelligenceNeural and Evolutionary Computing
Summary
Identifying what newly discovered proteins do is very slow because there are so many new protein sequences and not enough experts to label them. The authors found a way to teach a compact language model to write detailed protein function descriptions like expert curators do. They trained the model using a special fine-tuning method called QLoRA and tested it by having another AI act as an expert judge. Their results show this approach can produce useful biological annotations for many proteins.
What this means in practice
- •For bioinformatics engineers: Create automated tools that generate detailed functional descriptions for novel proteins beyond fixed label sets.
- •For pharmaceutical developers: Speed up early drug target identification by using AI-generated curator-style protein function annotations.
Authors
Demian Pavlyshenko, Bohdan Pavlyshenko
Abstract
Functional annotation of newly sequenced proteins remains a bottleneck in molecular biology: the number of sequences in public repositories grows far faster than the capacity for manual curation. Most computational approaches consider annotation as multi-label classification over a fixed ontology, which constrains predictions to a predefined label set. In this work we study the the protein annotation as a sequence-to-text generation problem. We fine-tune the 3B-parameter Ministral 3 base model with QLoRA (4-bit NF4 quantization with low-rank adapters) on sequence annotation pairs. We assess predictions with an LLM-as-expert protocol: a GPT model prompted as a senior molecular-biology curator scores organism identification as binary and function annotation quality. We conclude that QLoRA-fine-tuned compact LLMs can generate curator-style annotations with genuine biological value for a substantial subset of proteins. We also discuss future directions in data quality, model scaling, and evidence grounding that are needed to make the approach sufficiently reliable for practical use.