LDARNet: DNA Adaptive Representation Network with Learnable Tokenization for Genomic Modeling
2026-06-03 • Computation and Language
Computation and Language
AI summaryⓘ
The authors developed LDARNet, a genomic model that learns how to split DNA sequences into meaningful chunks on its own, instead of using fixed chunks like previous models. This adaptive chunking helps the model better understand important biological features such as promoter regions and splice sites without being told where they are. They tested LDARNet on many genetic tasks and found it often outperforms much larger models. Their experiments show that the advantage comes specifically from the model's ability to learn how to divide sequences dynamically.
genomic foundation modeltokenizationk-mersmasked language modelingBiMamba-2local attentionpromoter motifssplice junctionshistone modificationstate-space layers
Authors
Daria Ledneva, Denis Kuznetsov
Abstract
Genomic foundation models increasingly adopt large language model architectures, yet almost universally rely on fixed tokenization schemes such as $k$-mers, BPE, or single nucleotides, which impose arbitrary sequence boundaries that may obscure biologically relevant structure. We present LDARNet, a 120M-parameter hierarchical genomic foundation model that adapts H-Net-style dynamic chunking from autoregressive generation to masked language modeling, combining BiMamba-2 state-space layers with local attention, bidirectional routing, and a ratio-based regularizer to induce adaptive token boundaries without supervision. Fine-tuned on 27 tasks from the Nucleotide Transformer and Genomic Benchmarks suites, LDARNet achieves 11/18 wins among compact models ($<$300M parameters) and state-of-the-art results on 5 histone modification tasks, outperforming models up to 20$\times$ larger. A FLOPs-matched controlled experiment isolates learned routing as the source of these gains: learned boundaries beat fixed-grid boundaries by up to 14 percentage points on histone tasks at identical compute. Nucleotide-resolution analysis further shows that the learned boundaries align with canonical promoter motifs and splice junctions without supervision, providing a biological interpretation for adaptive tokenization in genomic foundation models.