Photodiagnosis with deep learning: A GAN and autoencoder-based approach for diabetic retinopathy detection - GENCER, Kerem; GENCER, Gülcan; CERAN, Tuğçe Horozoğlu; BİLİR, Aynur ER; DOĞAN, Mustafa (Photodiagnosis and photodynamic therapy, 2025)
•Our study introduces a novel approach integrating generative adversarial networks (GANs) to improve class distribution and generalization in diabetic retinopathy classification tasks, advancing beyond prior works that predominantly focused on segmentation or synthesis.•The dual-stage enhancement combining GANs and denoising autoencoders ensures high-quality synthetic and original retinal images by reducing noise and improving image clarity, thus delivering superior inputs for classification.•EfficientNetB0, a cutting-edge convolutional neural network (CNN), is fine-tuned on the enhanced dataset to achieve precise and accurate classification of optical coherence tomography (OCT) images, offering scalability and exceptional performance with minimal computational cost.•The uniqueness of our dataset, curated with high-resolution OCT images annotated by experts, distinguishes our work. Unlike publicly available datasets, our dataset emphasizes clinical relevance and ensures reliable benchmarking for diabetic retinopathy diagnosis.•By addressing key challenges such as noisy image inputs, class imbalance, and limited annotated data, our method provides a robust, scalable, and clinically applicable solution for diabetic retinopathy detection.•Our integrated framework of GANs, denoising autoencoders, and EfficientNetB0 sets our work apart from prior studies like GAN-Based retinal vasculature segmentation and autoencoder-driven multimodal Learning, emphasizing classification accuracy, dataset quality improvement, and optimized workflows for clinical photodiagnosis applications. Diabetic retinopathy (DR) is a leading cause of visual impairment and blindness worldwide, necessitating early detection and accurate diagnosis. This study proposes a novel framework integrating Generative Adversarial Networks (GANs) for data augmentation, denoising autoencoders for noise reduction, and transfer learning with EfficientNetB0 to enhance the performance of DR classification models. GANs were employed to generate high-quality synthetic retinal images, effectively addressing class imbalance and enriching the training dataset. Denoising autoencoders further improved image quality by reducing noise and eliminating common artifacts such as speckle noise, motion blur, and illumination inconsistencies, providing clean and consistent inputs for the classification model. EfficientNetB0 was fine-tuned on the augmented and denoised dataset. The framework achieved exceptional classification metrics, including 99.00 % accuracy, recall, and specificity, surpassing state-of-the-art methods. The study employed a custom-curated OCT dataset featuring high-resolution and clinically relevant images, addressing challenges such as limited annotated data and noisy inputs. Unlike existing studies, our work uniquely integrates GANs, autoencoders, and EfficientNetB0, demonstrating the robustness, scalability, and clinical potential of the proposed framework. Future directions include integrating interpretability tools to enhance clinical adoption and exploring additional imaging modalities to further improve generalizability. This study highlights the transformative potential of deep learning in addressing critical challenges in diabetic retinopathy diagnosis.