Papers for

biotechnology development teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Biosynthetic gene cluster benchmark compares multi-scale models and tasks

Data-Scarce, Multi-Granular, Multimodal: Benchmark Design for Biosynthetic Gene Clusters

Abstract: Biosynthetic gene clusters (BGCs) are co-located genes that encode the biosynthesis of specialized metabolites, an important source of antibiotics, anticancer agents, immunosuppressants, and agricultural bioactives. Deep learning, particularly self-supervised foundation models, has spurred interest in BGC sequence modeling, but evaluation has lagged behind: existing benchmarks often rely on coarse classification over small curated datasets, providing limited discrimination among models and insight into downstream utility. We propose a benchmark-design framework in which an informative benchmark should be discriminative, separating models with statistical confidence, and applicable, spanning scientifically meaningful tasks under different levels of generalization. We instantiate this framework as BGC-Bench, which derives five tasks from MIBiG, ranging from A-domain substrate prediction to BGC-molecule retrieval, evaluates them under genotype- and phenotype-shift splits, includes an active-learning setting for label-scarce discovery, and is easily extensible to new tasks and encoders. BGCs further provide a shared testbed across granularities and modalities: BGC-Bench compares twelve encoders on nucleotide, amino-acid, or Pfam-domain sequences, including compositional baselines for each input type, and combines BGC and chemical encoders for cross-modal retrieval. No single representation dominates: the best granularity depends on the biological scale of the task, and pretrained encoders do not consistently outperform other representations across tasks and splits. Active-learning experiments show that most tasks remain data-limited, encoder rankings change with labeling budget, and acquisition strategies yield modest gains. These results show that BGC model evaluation requires task-diverse, shift-aware benchmarks, and the framework can be generalized to benchmark design beyond BGCs.

Sun 1 NovArtificial Intelligence
The gist
Biosynthetic gene clusters (BGCs) are groups of genes that work together to produce useful molecules like antibiotics. The authors created a new testing system called BGC-Bench to better evaluate computer models that predict BGC functions and related molecules. This system uses various tasks that reflect real scientific challenges and compares different ways of representing BGC data, like DNA or protein patterns. Their findings show no single best way to model BGCs for all tasks, and that many tasks still need more data for better model training. This benchmark helps guide future improvements in using AI to understand and discover important natural products.
Open → 2611.02275v1

New benchmark reveals gene-specific limits in predicting cell responses

AmbiModBench: Benchmarking Gene Perturbation Prediction Beyond Shared Responses

Abstract: Predicting cellular responses to genetic perturbations helps prioritize experiments in single-cell genomics, where exhaustive measurement is infeasible. While computational models increasingly predict these responses, three evaluation deficiencies obscure what their scores demonstrate. First, absolute metrics cannot separate target-specific predictions from a shared background response. Second, common metrics remain high under gene shuffling, so gene-level accuracy is never verified. Third, a score at one training size says nothing about coverage, which depends on representation-space proximity and response-constraining power. We propose AmbiModBench, a specificity-aware, gene-resolved and coverage-aware benchmark. It pairs every score with a training-mean reference fitted on the same split, screens each readout by gene-coordinate permutation, and links embedding distance to response variation. Across K562, RPE1 and Norman, strong absolute scores largely reflect shared background rather than target-specific learning. Widely used readouts track response magnitude distributions rather than the affected genes. Detectable gain follows representation-space coverage rather than training-set size. Nonetheless, on RPE1 the protocol yields a reproducible target-specific gain across five additional splits and three gene selections, which absolute scores alone cannot distinguish from shared background.

Sat 26 SeptArtificial IntelligenceMachine Learning
The gist
Predicting how cells react when specific genes are changed is important but complicated because models sometimes just learn common reactions rather than gene-specific effects. The authors created AmbiModBench, a new way to test these predictions that separates shared patterns from unique gene effects and checks how well predictions cover different gene areas. Their tests showed that many existing methods score well by capturing general trends but fail to identify individual gene responses clearly. This benchmark helps find methods that truly understand gene-level effects, improving how experiments in single-cell genomics can be prioritized.
Open → 2609.32527v1