Deep learning algorithm finds specific patterns in nonlinear combinations of different data types, and manual feature extraction is not required. In recent years, deep learning algorithms have replaced traditional machine learning algorithms because the former are superior in mass nonlinear data [1]. The CNNs of Yann and Yoshua [13] are commonly used for computer vision tasks, including medical imaging, associated with high-performance image recognition. Since that time, many other valuable image recognition models have been developed and have shown good results [14–16]. Recently, CNNs performed as well as did ophthalmologists in terms of diagnosing diabetic retinopathy, age-related macular degeneration (AMD), and glaucoma; they also predicted patient age, sex, and cardiovascular disease status [16–18]. One study [7] found that CNNs could distinguish right and left color fundus photographs (mirror images) with 99.9% accuracy even after horizontal flipping. CNNs recognize differences between the left and right of the macula (humans do not). The color fundus photographs were obtained by exposing the entire retina to broad-spectrum white light followed by the capture of all reflected light. Monochromatic fundus photographs can be taken in various modes; the wavelength and filter can be varied. Various fundus structures are optimally visualized when the spectral range of the illuminating source is limited [19]. We explored whether CNN performance was reproducible even when the wavelength and filter varied. We tested RNFL, BAF, and IR monochromatic fundus photographs.
RNFL imaging is a monochromatic technique that uses a green contrast filter to correct individual tones. A color fundus photograph is a polychromatic image; the eye is illuminated with white light and examined in full color. RNFL images, on the other hand, are monochromatic, and afford better contrast than polychromatic images when viewing retinal vessels and their hemorrhages; pale lesions such as drusen; and exudates, nerve fiber layer defects, and epiretinal membranes [19]. Such RNFL photography reduces the red of the retinal pigment epithelium (RPE) and choroid and increases contrast; the white RNFL is more clearly seen [20]. At longer wavelengths, light scatters less; RNFL fundus photography affords good views even when the media are opaque [19].
The optic disc and retinal vessels of the right and the left eyes are mirror-symmetric. The optic disc evidences the greatest difference on the RNFL images of both eyes; even humans can distinguish the left and right eyes (Fig. 2A; the symmetricity test). CNNs distinguished the left and right sides with 100% accuracy (Test 1; AUC = 1.00, p < 0.001). In CAM of the RNFL image, the optic disc and the superior peripapillary area were important when distinguishing left and right (Fig. 2A). Next, we explored whether the right and left sides of the RNFL image could be distinguished even when the images were horizontally flipped (Fig. 2B; mirror symmetricity test). The human eye cannot distinguish between a right eye image and a flipped image of the left eye. However, the CNNs discriminated right and left with 97.93% accuracy (AUC = 0.98, p < 0.001) (Fig. 6). Previous studies on color fundus photography showed that the macula was allocated the highest weight on CAM images. CAM in this study also highlighted the superior peripapillary area, the macula, and the temporal perimacular area (Fig. 2B). The RNFL images revealed asymmetric differences between the right and left eyes in terms of the macula, temporal perimacular area, and superior peripapillary area.
Thus, it can be assumed that the CNNs identified the asymmetries of the superior peripapillary and temporal perimacular areas of both eyes in the RNFL images. These asymmetries have been reported previously. Cameron et al. [21] found that the RNFL thickness of the right eye was greater than that of the left eye, Jee et al. [22] reported that the locations of the superotemporal retinal artery and vein differed between the two eyes. Leung et al. [23] reported that the mean, central, retinal arteriolar equivalent of the right eye was 3.14 µm larger than that of the left eye. These differences cannot be discerned by humans. As the RNFL images afforded good contrast of both the RNFL and retinal blood vessels, asymmetries in terms of RNFL thickness and retinal blood vessel locations in the superotemporal and perimacular areas could be identified. It is thus possible that the superior peripapillary and perimacular areas were highlighted in CAM of the RNFL.
Fundus autofluorescence examination is a non-invasive technology that detects natural fluorophores that absorb and emit light of specific wavelengths [24]. High-speed BAF images were taken using an excitation wavelength of 488 nm and an emission spectrum of 500–700 nm [25]. RPE lipofuscin features multiple autofluorescent molecules that can be visualized using blue excitation light (BAF at 488 nm) [26]. BAF usefully evaluates various diseases of the retina and RPE, including degenerative, dystrophic, inflammatory, infectious, neoplastic, and toxic conditions [24].
The BAF set was prepared and analyzed as was the RNFL set. In Test 1 (the symmetricity test), the CNNs distinguished the left and right sides of BAF images with 99.74% accuracy (AUC = 0.99, p < 0.001) (Fig. 6). On CAM, both the optic disc and temporal perimacular area were highlighted (Fig. 3A). However, after the left-sided images were flipped horizontally (Test 2; mirror symmetricity test), the CNNs evidenced statistically significant (but nonetheless slight) lower accuracy than for the RNFL set (accuracy = 66.15%, AUC = 0.67, p < 0.001; Fig. 6). The macula was highlighted by CAM (Fig. 3B).
A possible explanation of the inferior results on the mirror symmetricity test is a distributional difference between the pigments that absorb or emit blue light in the BAF image. In BAF images of normal eyes, the gray optic nerve head becomes dark black given the absence of lipofuscin. Similarly, blood vessels exhibit weak BAF signals because hemoglobin absorbs blue light [27]. Macular pigments (lutein, zeaxanthin, and meso-zeaxanthin) absorb the short-wavelength excitation light of BAF imaging. The foveola is hypoautofluorescent because of blue light absorption by luteal pigments and melanin. Therefore, foveolar evaluation is challenging in BAF images [26]; accuracy is reduced by central shadowing attributable to macular pigment.
IR (815 nm) imaging is performed prior to indocyanine green angiography or fluorescein angiography; near-infrared light (not white light) is used to evaluate the RPE and the underlying choroidal structures [28]. IR photography is a variant form of fundus photography using an excitation wavelength of 820 nm without a cut-off filter [29]. IR imaging affords several advantages compared to flash fundus photography. IR light is invisible; this aids the imaging of children and light-sensitive patients. The high-level IR transmittance renders this photographic mode valuable when examining patients with media opacities (e.g., dense cataracts). IR images better reveal the retinal epithelium, cystic macular edema, and deep structures than do fundus photographs and images lacking redness. IR was superior to standard fundus photography when screening for neovascular AMD [19].
The results of the IR set were similar to those of the RNFL set. In Test 1 (the symmetricity test), the CNNs exhibited 100% accuracy (AUC = 1.00, p < 0.001) (Fig. 6). On CAM, the optic disc, superior peripapillary area, and superotemporal perimacular area were highlighted (Fig. 4A). In Test 2 (mirror symmetricity test), the accuracy was 95.84% (AUC = 0.96, p < 0.001; Fig. 6) and the superior peripapillary and superotemporal perimacular areas were strongly highlighted (Fig. 4B). In the mirror symmetricity tests of the RNFL, BAF, and (particularly) the IR sets, CAM highlighted the superior peripapillary area more than the inferior perimacular area. It is known that RNFL asymmetries include those of the superotemporal retinal arteries and veins [21, 22]. IR imaging may better recognize these differences than do other imaging modalities. Alternatively, there may be an (as yet unknown) anatomical difference between the left and right superior areas of the fundus.
Test 3 (the overfitting test) was prepared to confirm that the CNNs detected differences between the right and left eyes. If the CNN results were attributable to (undetectable) overfitting, thus not real differences between the right and left images, the CNN accuracies would be similar after training using one-sided images only. Conversely, if the CNNs found real differences, these would not be detected after training on one-sided images. We prepared two classes (subsets 1 and 2) of only left eye images. Given that there were two classes, the accuracies should be about 50%. All three Test 3 accuracies were indeed very close to 50% (Fig. 6) and the CAM patterns revealed no regularity (Figs. 2C, 3C, and 4C). These results support the results of Tests 1 and 2; the CNNs identified real differences between the right and left eyes.
The limitations of the study include the fact that it was a single-center work and the numbers of RNFL, BAF, and IR images differed. Verification is required using other imaging equipment at several institutions. A previous study used lossy compression algorithms JPG file format [7]. A strength of our work is that BMP files are lossless. In summary, CNNs distinguish the right from the left eye by identifying macular, and peripapillary and temporal perimacular areal, asymmetries that humans cannot see, confirming the macular asymmetry described in an earlier, color fundus photographic study [7]. CNN results vary by the imaging modality. Using BAF, it is difficult to accurately evaluate the macula given the intrinsic image characteristics; CNN accuracies were poor on the mirror symmetricity test. Therefore, in future, caution is appropriate if using BAF to identify the macula. Also, as right and left inverted image bias may be in play among the various fundus photographic modes, care should be taken in building the CNN model.