DNN Classifier of Wide-angle Retinal Images in Computer-Aided Screening for ROP
Background: Retinopathy of prematurity (ROP) is one of the main causes of childhood blindness. However, insufficient ophthalmologists are qualified for ROP screening.
Objective: To evaluate the performance of a deep neural network (DNN) for automated screening of ROP.
Methods: The training and test sets came from 420,365 wide-angle retina images from ROP screening. A transfer learning scheme was designed to train the DNN classifier. First, a pre-processing classifier separated unqualified images. Then, pediatric ophthalmologists labeled each image as either ROP or negative. The labeled training set (8090 positive images and 9711 negative ones) was used to fine-tune three candidate DNN classifiers (AlexNet, VGG-16, and GoogLeNet) with the transfer learning approach. The resultant classifiers were evaluated on a test data set of 1742 samples, and compared with five independent pediatric retinal ophthalmologists. The ROC (receiver operating characteristic) curve, ROC area under the curve (AUC) and P-R (precision-recall) curve on the test data set were analyzed. Accuracy, precision, sensitivity (recall), specificity, F1 score, Youden index, and MCC (Matthews correlation coefficient) were evaluated at different sensitivity cutoffs. The data from the five pediatric ophthalmologists were plotted in the ROC and P-R curves to visualize their performances.
Results: VGG-16 achieved the best performance. At the cutoff point that maximized F1 score in the precision-recall curve, the final DNN model achieved 98.8% accuracy, 94.1% sensitivity, 99.3% specificity, and 93.0% precision. This was comparable to the pediatric ophthalmologists (98.8% accuracy, 93.5% sensitivity, 99.5% specificity and 96.7% precision).
Conclusion: In the screening of ROP using the evaluation of wide-angel retinal images, DNNs had high accuracy, sensitivity, specificity, and precision, comparable to that of pediatric ophthalmologists.
This repository contains the following files: 1. data set preview.png A preview of the data set. 2. Training Set Thumbnails.zip The training dataset includes: Class I (with disease): 8089 Class II (without disease): 9711 * This is a snapshot of the resized (80x60) version of the training & test data set. The original image data set are not provided, due to patient privacy protection and hospital IRB regulations. 3. Test Set Thumbnails.zip The test training dataset includes: Class I (with disease): 155 Class II (without disease): 1587 * This is a snapshot of the resized (80x60) version of the training & test data set. The original image data set are not provided, due to patient privacy protection and hospital IRB regulations. 4. DNNs.zip This zip file contains 3 subfolders: AlexNet, GoogLeNet, VGG16. Each subfolder further includes: Model definition graph Two trained models in the Caffe format. One is trained "from scratch" and the other "by transfer learning". P-R curve ROC curve Softmax output probabilities on the test data set * We tried many times but failed to upload the zip (1.41GB). Please access this file on figshare (https://figshare.com/articles/DNN_Classifier_of_Wide-angle_Retinal_Image...) 5. HumanExpertsPerformance.zip This zip file contains 3 subfolders: AlexNet, GoogLeNet, VGG16. Each subfolder further includes: Model definition graph Two trained models in the Caffe format. One is trained "from scratch" and the other "by transfer learning". P-R curve ROC curve Softmax output probabilities on the test data set 6. scripts.zip Scripts used in this study for batch inference and data analysis. 6. Response to reviewers.pdf. Response letter to reviewers that provides additional materials.
- datasetpreview.png (815.49 kB)
- TrainingSetThumbnails.zip (33.42 MB)
- TestSetThumbnails.zip (10.96 MB)
- HumanExpertsPerformance.zip (248.73 kB)
- scripts.zip (20.66 kB)
- response to reviewers.zip (1.15 MB)