Panvram enables fast GPU access to human pangenome data cohorts
panvram: GPU-resident random access to a human pangenome cohort
Distributed, Parallel, and Cluster ComputingPerformance
Summary
Accessing many variations of human genetic sequences quickly and efficiently is difficult because of their size and complexity. The authors present panvram, a system that stores many human genomes compactly on a graphics processing unit (GPU), allowing very fast reads of short sequence parts. This system achieves much higher access speed compared to traditional CPU methods while using modest memory. It measures actual performance on hundreds of genomes and provides tools to predict access costs. The work helps manage large-scale genetic data and supports faster analysis for genetics research.
What this means in practice
- •For bioinformatics software teams: Enable fast retrieval of sequence windows from large human pangenome cohorts using GPU-resident compressed data structures.
- •For computational genomics platform developers: Integrate efficient GPU-based human genome pangenome storage to accelerate cohort-scale genomic analyses.
Authors
Yakiv Shavidze
Abstract
Repeatedly sampling short sequence windows from a human pangenome requires an access representation as well as compression. panvram stores each assembly as a reference-relative block archive and pools these archives on a GPU. For 558 haplotype assemblies, the default q4k representation occupies 16.642 GB of persistent pools including the decoded reference; a packed-reference, compact-directory representation occupies 11.634 GB. These quantities are not peak application memory. In the C1 experiment, default and fully compact forms delivered 743,271 and 736,762 windows/s; a separate public Colab run delivered 741,931 and 680,760 windows/s. All are batch API measurements and must not be pooled as repeated trials of an identical environment. Full-decoding checks compared reconstructed FASTA checksums with the manifest. On the CPU, the M6 shuffled-window workload at the same cohort cardinality reported 26 requests/s for AGC and 145,222 requests/s for refrel3 q4k. This comparison concerns one-thread requests of 4 KiB on the recorded warm-cache workload, not all genome-access patterns. AGC provides the smaller incremental archive: 5.40 MB/assembly versus 13.04 MB/assembly for refrel3 q16k, with different exclusions; the size pair is not the same refrel3 configuration as the q4k timing pair. We derive an expected-work expression for fixed independent granules and distinguish it from an empirical median-latency predictor. Cohort-scale counter and DRAM measurements motivate a locality hypothesis, but do not establish a causal per-miss latency or independently reproduce a model-failure verdict. The window-cost verdict at N=558 is published with its harness and raw timings.