Medical images translated without paired training while preserving anatomy

Anatomy-Preserving Unpaired Medical Image Translation via Shared Latent Anchoring

Computer Vision and Pattern Recognition

Summary

Converting medical images from one type to another usually requires matched pairs of images for training, but this is hard to get. The authors propose a method that separates learning the body’s anatomical structure from adding the new image appearance, which helps keep important details intact. They first teach a model to understand body structures across image types, then generate new images based on this stable structure. Tests on eye and brain scans show this approach produces clearer, more accurate translated images than existing methods, even those trained with pairs.

What this means in practice

  • For medical imaging software developers: Create systems that convert scans from one imaging type to another without needing paired datasets, while keeping anatomical accuracy.
  • For medical device companies: Improve image-based diagnostic tools by generating complementary scan types that highlight anatomy without distorting structures, even when paired data is unavailable.$Commercial implications: Enables selling enhanced imaging devices with better cross-modality image synthesis for hospitals and clinics.

Authors

Zhilin Zou, Hieu Le, Saumya Gupta, Jingyi Xu, Prateek Prasanna, Chao Chen

Abstract

Unpaired medical image-to-image translation requires synthesizing target-modality images from source-modality images without aligned supervision. Existing methods learn a direct source-to-target mapping that entangles structure preservation and appearance generation within a single model, often distorting clinically meaningful anatomy in the translated output. We argue that these two objectives should be decoupled entirely: anatomical structure should be captured independently of generation, and the generator should operate only on a frozen structural representation. To this end, we propose a two-stage framework for unpaired medical image translation. In the first stage, we learn a shared structure space by training a structure encoder-decoder on source images, target images, and structural masks jointly, using mask reconstruction to anchor the latent space to modality-agnostic anatomy. For source images without mask annotations, a target-domain memory bank provides soft structural supervision via prototype retrieval. In the second stage, a conditional flow model learns to render target-domain appearance under fixed structural conditions. This ensures the generator inherits target appearance statistics and can focus entirely on structural conditioning rather than learning appearance from scratch. We validate our framework on OCT-to-OCTA synthesis and brain MR-to-CT translation. Our method outperforms all unpaired baselines on both tasks and, despite requiring no paired supervision, also surpasses paired baselines, improving PSNR by up to 11.2% and SSIM by up to 26.4% on OCT-to-OCTA, and achieving consistent gains across whole-image, soft-tissue, and bone regions on MR-to-CT. The project repository is available on GitHub.