Papers for

cybersecurity teams in biotech

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Adversarial attacks can poison protein design data and outputs

Poisoning the corpora of protein design systems during pre-training and retrieval

Abstract: Protein foundation models increasingly support the generation of novel biological sequences, yet their reliance on large public datasets creates opportunities for adversarial manipulation of the material they learn from and retrieve. We investigate two data-only poisoning attacks against protein design systems, targeting the pre-training corpus of autoregressive protein language models and the retrieval corpus of a text-guided protein generator. For the first, we pre-trained ProtGPT2 and ProGen2 variants of 300 million parameters, introducing poisoned protein sequences containing one of two triggers followed by one of three payloads. Across all poisoned runs, both architectures acquired highly reliable trigger-dependent generation under uniform and ramped poisoning schedules. Under ramped exposure, the earliest majority activation occurred after only 200 contaminated examples, while for the uniform schedule earliest activation occurred around 5000 poisoned samples. For the second attack, we inserted records into a retrieval corpus system without modifying its weights or decoding procedure. A single query-aware record captured a top-16 retrieval position for 58\% of templated requests and 97\% of free-text requests. Conditional on successful poisoned retrieval, 62\% of requests produced altered sequences, while 19-21% acquired the inserted domain by profile-HMM homology. A universal query-unaware attack using 1,500 records also achieved substantial retrieval capture and domain transfer. Together, these findings demonstrate that adversarial protein sequence contributions can establish persistent, sequence-specific behaviour during pre-training or redirect protein generation at inference, while leaving ordinary model behaviour largely preserved. Our work establishes the integrity and biological provenance of protein sequence data as critical safeguards for generative protein design systems.

Thu 15 OctMachine Learning
The gist
Protein design tools learn from large collections of protein sequences to create new proteins. The authors show that attackers can sneak in harmful or misleading sequences during the training or retrieval phases, which makes the models produce specific altered proteins when triggered. These attacks can work with surprisingly few poisoned examples and often go unnoticed because the models still behave normally otherwise. This highlights the need to carefully check and secure protein data used in these systems.
Open → 2610.21200v1