Papers for

biomedical imaging developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Vision transformer joins images and gene data for better tissue analysis

HistoTx: Early fusion of H&E images and spatial transcriptomics at varying spatial transcriptomics resolution with self-supervised learning

Abstract: Spatial transcriptomics maps gene expression across tissue sections and, when integrated with whole slide images, jointly captures both morphological architecture and molecular activity to provide a rich, multimodal view of tissue biology. ML approaches combining spatial transcriptomics and WSIs typically learn a mapping from morphological features to molecular profiles via contrastive alignment or supervised fine-tuning of pathology foundation models. In both cases, transcriptomics serves as a training signal rather than a true input modality, leaving inference fundamentally image-driven. We propose an early-fusion vision transformer architecture in which transcript tokens are constructed at arbitrary spatial resolution, merged with image patch tokens, and jointly processed through a shared transformer stack for deep cross-modal interaction. Starting from a pretrained vision-only pathology foundation model, we train HistoTx by continuing self-supervised pretraining on paired image and spatial transcriptomics data and demonstrate that: 1) at inference with image only, performance remains on par with the baseline on most tasks while improving on tasks that inherently benefit from molecular knowledge, such as gene expression prediction; 2) when both modalities are available at inference, jointly providing image and transcript inputs outperforms either modality alone.

Mon 2 NovMachine LearningArtificial Intelligence
The gist
Understanding tissue biology is easier when looking at both pictures of tissue slices and their gene activity. The authors created a new computer model that combines these two kinds of information early on, letting the model learn from both together. This approach works well even when only pictures are available later and works even better when gene data can be added at the same time. Their method helps better predict molecular details from images and improves overall analysis when both data types are used.
Open → 2611.02681v1

Deep learning speeds up electron microscopy brain imaging dramatically

ZoomSEM: accelerating large-scale brain connectomics with intelligent SEM super-resolution

Abstract: Serial electron microscopy (EM) is essential for large-volume biological imaging and connectomics, yet its throughput remains a major bottleneck, with cubic-millimeter-scale brain imaging requiring months or years. Existing deep learning-based EM super-resolution methods are limited to small upscaling factors or rely on synthetic data, offering modest acceleration and limited practical impact. We introduce ZoomSEM, a deep learning-based super-resolution method to dramatically accelerate SEM imaging. Trained on real low- and high-resolution scanning electron microscopy (SEM) pairs, ZoomSEM enables reliable recovery of 4$\times$, 8$\times$, and up to 16$\times$ higher-resolution images from fast low-resolution scans while preserving fine neuronal structures critical for connectomic analysis, achieving up to 30-fold effective acceleration. Extensive experiments across datasets of diverse biological samples demonstrate robust generalization under realistic low-resolution acquisition. This advance enables high-resolution, large-volume serial EM imaging at unprecedented speed, poised to transform connectomics and related fields.

Sat 24 OctComputer Vision and Pattern Recognition
The gist
Imaging the brain in extremely high detail using electron microscopes takes a very long time, sometimes months or years. The authors created ZoomSEM, a smart computer method that takes quicker, lower-resolution scans and turns them into much sharper images, preserving tiny brain details needed to study how neurons connect. This method can make the scanning process up to 30 times faster. It works well on various biological samples, promising to speed up brain research and other areas that need detailed imaging.
Open → 2610.33580v1