Improving skin cancer AI training with safer synthetic images
Evaluating Synthetic Skin Lesion Images for Classifier Training: A Safety-Aware Algorithm
Computer Vision and Pattern RecognitionMachine Learning
Summary
Skin cancer detection AI needs many pictures of skin lesions, but there are not enough real ones, especially for rare types. The authors studied computer-generated images to help train these AIs but found that many quality measures used to pick good images don’t work equally well for all skin lesion types. They created a new method that checks multiple quality scores and removes images that look too much like real photos to keep things safer. Their method slightly improved AI performance in tests, offering a more careful way to use synthetic data for skin cancer detection.
What this means in practice
- •For medical ai developers: Select safer, higher-quality synthetic images to train skin lesion classifiers and improve detection performance on rare lesion types.
- •For healthcare data teams: Curate synthetic dermatology image datasets to reduce risks from reused real patient images while maintaining classifier accuracy.
Authors
Amindu Kumarasinghe, Vinh Bui, Reza Ghanbarzadeh
Abstract
Skin cancer classifiers need large sets of annotated images, which are scarce, especially for rare lesion types. AI-generated images can fill the gap, but a generator produces images of widely varying quality, so selecting which images to retain is important. That choice usually rests on a single quality score, most often the Fréchet Inception Distance (FID), which has not been tested against what a classifier needs and cannot tell whether the generator memorizes its training photographs. We test that practice with a tournament of 16 quality metrics. Each metric alone selects a training subset from a pool of 275,472 images generated by a diffusion model (EDM2), and a classifier trained on each subset scores the metric per class. The tournament shows that no single metric serves every class: FID computed on Inception features gives the best classifier for one lesion type and no measurable advantage over random sampling for another, and the generation setting it prefers is also the one closest to the training images. We turn these per-class measurements into a safety-aware evaluation algorithm that weights each metric by its measured value in each class and removes images too close to a real training image. On the held-out ISIC-2018 test set, the algorithm improves Macro-$F_2$, a recall-weighted measure of classification performance, over the FID baseline for a Swin-B classifier by +0.0136 to +0.0164 across four safety thresholds (Bonferroni-corrected $p < 0.02$) and over random sampling by about +0.03, though not for ResNet-50. Tightening the threshold improves a memorization-safety score with no measurable change in utility, but every curated set remains less safe than random sampling. The results show a modest but statistically significant gain on the primary classifier, and offer a practical, safety-aware procedure for evaluating and using synthetic data.