Machine learning models assessed for skin lesion diagnosis across devices and groups
Bridging Research and Practice: A Systematic Evaluation of Generalist and Dermatology-Specific Models in Clinical Skin Lesion Classification
Computer Vision and Pattern Recognition
Summary
Skin conditions can look very different depending on who has them and how the pictures are taken, making it hard for computer programs to diagnose them accurately everywhere. The authors tested many different types of artificial intelligence models to see how well they work on skin images from various devices and patient groups. They compared general-purpose AI models with ones designed just for skin diagnosis to understand how reliable these models are when used in real clinics. This work helps show what is needed before AI can be safely used to assist doctors in diagnosing skin problems.
What this means in practice
- •For clinical ai developers: Improve and validate AI tools for diagnosing skin lesions using diverse real-world images and patient groups.
- •For medical device manufacturers: Design imaging devices that capture dermatology photos aligned with AI models’ strengths to improve diagnosis accuracy.
Authors
Emanoel dos Santos, Kelvin Cunha, Rodrigo Mota, Fabio Papais, Thales Bezerra, Natalia Lopes, Erico Medeiros, Shirley Cruz, Jessica Araujo, Paulo Borba, Tsang Ing Ren
Abstract
The application of machine learning to dermatology has grown substantially in recent years, moving beyond proof-of-concept studies toward potential applications. However, clinical dermatology remains a challenging and still open problem. Diagnostic assessment is often ambiguous, and skin lesions exhibit high variability, compounded by differences in acquisition modality, device quality, and patient demographics. These factors hinder the development of robust models suitable for safe and equitable clinical use. To support translation into practice, it is essential to systematically evaluate how contemporary models generalize across heterogeneous data sources. In this work, we benchmark a diverse set of architectures on recent dermatology datasets, spanning dermoscopic images and smartphone-based clinical photographs. We assess the robustness of recent general-purpose and medical vision-language models, as well as foundation models, and compare them against task-specific dermatology classifiers, including embedding-based approaches and convolutional neural networks. Our study provides an evaluation of model performance under distribution shifts, modality changes, and demographic variability. By quantifying the gap between current state-of-the-art models and the requirements of clinical deployment, we aim to contribute to the development of reliable, accessible, and clinically applicable AI systems for dermatology.