No Loss of Chance of Diabetic Retinopathy Screening by Endocrinologists With a Digital Fundus Camera

The local ethics committee approved the study protocol, and all patients signed an informed consent.

Five hundred consenting and able-to-cooperate adult patients referred to the diabetes department of the University Hospital of Saint-Etienne for diabetes during a 6-month period underwent a systematic DR screening. Medical details of each patient were recorded, including age, duration of diabetes, type of treatment, associated systemic risk factors, and history of eye treatment. All 500 patients underwent three-field retinal photography with the Topcon TRC NW6S digital camera (Topcon Europe, Rotterdam, the Netherlands) linked to a high-resolution (6.17 million pixels) Fujifilm Fine Pix S2 Pro Super CCD camera (Fujiphoto, Tokyo, Japan) in Tagged Image File Format (TIFF)-RGB recording mode with a density of 2,034 × 1,728 pixels. As previously described (7), retinal photography was performed after instillation of one drop of tropicamide 1% in each eye. Three 45-degree images of horizontal overlapping fields were captured (one central [the macula including the optic disk], one temporal [macula on the nasal edge], and one nasal [disk on the nasal edge]), and stored after compression (1:17) in JPEG format (16). Images were taken using the internal and, on rare occasions, external fixation target. The image acquisition process was repeated if the original image was judged unsatisfactory by the photographer. Automatic mosaic reconstruction was performed with ImageNet2000 software (Topcon Europe, Rotterdam, the Netherlands), followed by manual retouch to enhance the gradeability of partially obscured yet interpretable images. All the images were taken by the same photographer throughout the study.

Five endocrinologists and two ophthalmology residents underwent specific training over 40 h conducted by a consultant ophthalmologist specializing in retinopathy. The training consisted of both theoretical and practical sessions and was focused on photographic detection and gradation of DR including diabetic maculopathy (DM). Additional training was offered to detect associated fundus findings: hypertensive retinopathy (arteriolar narrowing, arterio-venous crossing, cotton wool spots, and flame shaped hemorrhages), age-related macular degeneration (ARMD), myopic chorioretinal degeneration, optic disc disorders such as edema, cupping, and atrophy, and choroidal lesions such as nevi and laser scars. Particular attention was paid to the criteria and the timing of referral to the ophthalmologists (4): noninterpretable images, severe DR stage >1, DM, papillary cupping, papillary atrophy, choroidal lesions, nevi, and all other unknown incidental findings. Upon completion of training in screening and grading the retinal images and satisfactory evaluation, the endocrinologists and ophthalmology residents were inducted into the study. A simplified DR classification adapted from American Diabetes Association and Francophone Diabetes Society recommendations was used for grading (4,5): stage 1 (mild nonproliferative DR [NPDR]) was represented by occasional microaneurysms and/or hemorrhages and/or exudates; stage 2 (moderate NPDR) by intra retinal hemorrhages and/or cotton-wool spots and/or venous anomalies in one to three quadrants; stage 3 (severe NPDR) by intra retinal hemorrhages and/or cotton-wool spots and/or venous anomalies in all quadrants; stage 4 (noncomplicated proliferative DR [PDR]) by detection of new vessels on disc or retina; stage 5 (complicated PDR) by tractional retinal detachment, preretinal or vitreous hemorrhage; DM by the presence of hard exudates within one disk diameter of the fovea, considered treated with the presence of photocoagulation scars anywhere (sectored, pan-retinal, focal, and grid). The consensus opinion of two retina specialists using the same three-field images evaluation was considered as the “gold standard.”

A total of 1,000 sets (500 patients, 1,000 eyes) of three images and the respective mosaics were recorded on a CD-ROM. Each clinician independently analyzed each of the 1,000 separate sets of images in a masked fashion (without knowing which the paired eyes were and without any clinical information) on 15-inch TFT screens of personal computers at a resolution of 1,400 × 1,050 pixels and 32-bit true-color display with the Windows XP image viewer (Microsoft Corporation, Redmond, WA) as previously described (7,17).

The observers had to note the following points for each set: 1) image quality of the individual images as well as reconstructed mosaics; 2) confidence degree for screening and grading DR depending on the quality of the images. Presence of DR, its grading, and presence of MD; and 3) presence of associated ocular findings, as previously described.

Sensitivity and specificity were evaluated for each observer for screening and grading of DR, including DM. Eyes with laser images or with noninterpretable images were referred directly to ophthalmologists. This group was used in the calculation of loss of chance. The agreement between the observers and the gold standard for the assessment of the digital images was expressed as an underweight κ index or κ value (18). Agreement of screening was classified as follows: almost perfect, κ >0.8; substantial, κ 0.8–0.6; moderate, κ 0.6–0.4; fair, κ 0.4–0.2; slight, κ 0.2–0.0; no agreement, κ <0.0. κ values were compared between endocrinologists and ophthalmology residents by a nonparametric Mann-Whitney U test (P < 0.05 considered significant). Eyes needing ophthalmologist referral for further management (referral criteria), but missed by endocrinologist and ophthalmology resident screeners were counted as loss of chance. Statistical analysis was performed by SPSS version 11.0 (SPSS, Chicago, IL).

Of the 500 patients, 263 (52.6%) were male. Type 1 diabetes was found in 121 patients (24.2%) with a mean age of 40 ± 22 years and a mean of 15 ± 15 years since disease onset. Type 2 diabetes was found in 368 (73.6%) patients who had a mean age of 64 ± 17 years and a mean of 12 ± 12 years since disease onset. Gestational diabetes was found in six patients (1.2%) and impaired glucose tolerance in five patients (1%). Type 2 diabetic patients were treated as follows: insulin therapy (66%), oral antidiabetic drugs only (28.7%), insulin and oral antidiabetic drugs (5.8%), or diet only (4.5%).

The flow diagram of the eyes/findings through the study is presented in Fig. 1. Out of 1,000 eyes, 38 showed noninterpretable images, and were therefore excluded from the retinal assessment and directly referred to the ophthalmologists. Of 962 eyes with analyzable images, DR was detected in 456 (47.4%). It consisted of 246 (53.9%) cases of mild NPDR (stage 1), 98 (21.5%) of moderate NPDR (stage 2), 14 (3.1%) of severe NPDR (stage 3), eight (1.8%) of uncomplicated PDR (stage 4), zero cases of complicated PDR (stage 5), 74 (16.2%) cases of inactive PDR post laser, and 16 (3.5%) cases of active PDR post laser. DM was detected in 186 eyes (2.3%). Associated ocular findings were found in 669 eyes (67%), and most frequently this consisted of hypertensive retinopathy (22%), ARMD (5.7%), and transparency media abnormalities (13%).

Out of the five endocrinologists, four had substantial agreement with the gold standard regarding appreciation of the image quality (κ = 0.69, 0.62, 0.63, 0.75, and 0.22). The agreement was substantial (κ = 0.73) for one ophthalmology resident and almost perfect (κ = 0.83) for the other.

The ability of all the endocrinologists and ophthalmology residents to detect laser on retina was almost perfect (κ >0.8).

Ninety eyes with laser images were not included in the analysis of screening accuracy. Thirty-eight eyes with noninterpretable images were not included in retinal assessment either. The screening accuracy, which demonstrates the agreement between each observer and gold standard, was evaluated on the remaining 872 eyes. κ index was substantial for three endocrinologists, moderate for one, and fair for one. The adjusted κ remained substantial for the same endocrinologists (κ = 0.78, 0.71, and 0.75) and moderate (κ = 0.4 and 0.6) for the rest. The average weighted κ was substantial for the endocrinologists (κ = 0.65 [range 0.55–0.68]). The κ index and the weighted one were substantial for both ophthalmology residents (weighted κ = 0.73 [0.65–0.70]) (Table 1). The average sensitivity of the endocrinologists was 89.7% [84.1–100.0]. The average sensitivity of the ophthalmology residents was 96.4% [95.1–97.8]. The negative predictive value of the screening was 91.9 [88.8–100.0] for the endocrinologists and 96.7 [95.2–98.0] for the ophthalmology residents. Out of 456 eyes with DR, 36 (8.25%) have been missed by the endocrinologists. All of them presented with mild NPDR (stage 1), which does not require referral. Fourteen cases (2.85%), all in stage 1 NPDR too, have been also missed by ophthalmology residents.

The grading accuracy was evaluated on 872 eyes. One hundred fifty-six eyes (10.2%) were in stage 2 NPDR, and 22 (2.3%) were in stage 3 or 4 NPDR (14 and 8, respectively). The agreement for DR grading between endocrinologists and gold standard was fair for two of them (κ = 0.40 and 0.35), moderate for two (κ = 0.44 and 0.49), and substantial for one (κ = 0.62). The κ index was substantial for the ophthalmology residents (κ = 0.76). The endocrinologists have under-classified as stage 2 NPDR several stage 3 NPDR cases (2, 6, 8, 10, and 10 respectively for each endocrinologist on 22 cases).

The agreement between the endocrinologists and gold standard for maculopathy screening was almost perfect for two of them (κ = 0.78 and 0.75), substantial for two (κ = 0.68 and 0.70) and moderate for one (κ = 0.53). The agreement between the ophthalmology residents and gold standard was almost perfect (κ = 0.83 and 0.77). Out of 186 eyes with DM, 20 eyes have been missed by both endocrinologists and ophthalmology residents.

The agreement between the observers and gold standard is presented in Table 1. The endocrinologists have frequently misdiagnosed ARMD as diabetic macular disease. The nonparametric comparison tests indicate trends for reduced efficacy in detecting associated findings for the endocrinologists compared with the ophthalmology residents.

The loss of chance concerning the screening performed by the endocrinologists and ophthalmology residents is presented in Table 2. This loss of chance was calculated from the 555 diagnoses to refer to the ophthalmologists according to the gold standard: 186 DM cases, 90 laser cases, 38 noninterpretable images, 120 DR stage >1, and 121 incidental findings. There was no loss of chance for the 120 eyes with DR to be referred (stage >1) for both endocrinologists and ophthalmology residents because none of these diagnoses has been missed.

Regarding the detection of DM, laser marks, noninterpretable images, and other associated findings, the difference of loss of chance between endocrinologists and ophthalmology residents was not significant. The endocrinologists missed significantly more papillary cupping and choroidal lesions (15.2 vs. 4.5 on 20 cases and 1.8 vs. 0.5 on 2 cases, respectively, P < 0.05).

This study compares the efficacy and the degree of loss of chance during screening performed by trained endocrinologists in comparison with ophthalmology residents. The average sensitivity of detection of DR by endocrinologists was good (89.7%), with only stage 1 NPDR disease having been missed. This has little consequence in terms of loss of chance on patient management because rapid ophthalmologic interventions are not required at this stage. Furthermore, these missed cases are likely to be picked up during subsequent annual examinations (4,5).

The endocrinologists also had difficulties in grading DR with several stage 3 NPDR diseases being under classified as stage 2. Those misdiagnoses do not amount to any loss of chance because the timing of referral to ophthalmologists for further management remains the same as per recommendations (4,5).

We did not find any major difference between endocrinologists and ophthalmology residents for DM detection and, consequently, no significant “loss of chance.” The results show a trend of a lower efficacy in screening associated ocular findings like nevi or optic disc atrophy. The low agreement for the incidental findings arising as a result of confusion between DR, hypertensive retinopathy, and ARMD can be explained by the confounding vascular changes occurring in these pathologies. The fact that the endocrinologists could detect these subtle findings demonstrates their good sensitivity. However, all of these patients were referred to the ophthalmologists, without loss of chance.

Overall, because there is no loss of chance, the screening of the DR performed with a digital fundus camera by trained endocrinologists was as accurate and safe as that performed by ophthalmology residents.

We previously reported that dilatation significantly improved image quality and certitude of screening DR using a nonmydriatic camera without disturbing the unit’s organization (20 min to obtain the dilatation, whether planned or not, no side effect and 5 min to read the images) (7). In the current study, after systematic dilatation, almost 80% of images sets presented with good quality. The agreement between endocrinologists and gold standard on the image quality was substantial (κ = 0.61). The comparable confidence level allows trained endocrinologists to screen with accuracy, and to decide to refer to the ophthalmologist in case of poor image quality, which ensures a safe delegation with no “loss of chance.”

Besides, we have found 555 diagnoses to refer on those 1,000 eyes in an exhaustive screening of each patient attending the care unit (Fig. 1). Obviously, many patients may have several coexistent pathologies, such as DR and hypertensive retinopathy, which is in agreement with our previous study. This exhaustive screening led to discover coexistent fundus pathologies, which appears to be of cost-effective second benefit (19). Additionally, this crossover diagnosis allows multiplying the chance of being referred for an eye, thus decreasing the loss of chance if one diagnosis has been missed.

Three specific points can be noticed: the device features, the delegation, and the location. Although the DR screening by endocrinologists using the direct ophthalmoscopy reported good results (14), this technique is less sensitive than the digital camera (20). The number of images to use for screening with digital camera is another issue to discuss. Current recommendations propose the use of two images (central and nasal focused) as sufficient for safe screening (4). Other recent data indicate that one central image would be sufficient to screen but not to grade (21). However, in a recent study the use of a single image by primary care clinicians failed to refer 10.2% of the patients who would have otherwise needed referral to the ophthalmologist (22). In our study, we used three images per eye to screen and grade with no loss of chance when compared with senior ophthalmologists. However, when comparing three versus seven-field comprehensive exploration, a minimal loss of chance could exist even for senior ophthalmologists (κ = 0.88) (21). The possibility of such a delegation among medical professionals has been considered since the 1990s, involving mainly general practitioners (22,23), optometrists, and orthoptists (24). There are numerous studies that demonstrate the utility of a computer program to help nonophthalmologists in screening (25), and telemedicine is well developed (11). In our study, screening has been performed by endocrinologists, who are well versed with the disease and risk factors, which could contribute to screening efficiency without disturbing the unit organization (additional 5 min for the screening performed by the endocrinologists). In contrast to previous studies involving screening by a single endocrinologist (14,15), our study has been performed by five endocrinologists with reproducible and similar skills. This screening has been performed in the diabetic care unit and not in general health centers, which allows the maximum possible recruitment of diabetic patients. Use of the digital fundus camera improves the rapidity and precision of screening thus allowing exhaustive coverage feasible in a hospital setting, as shown in our previous study (17). In addition, contrary to telemedical systems, the patients do not have to wait for the ophthalmologist’s diagnosis, with a complete diabetes work-up, glycemic control, and complications screening in a single appointment. The endocrinologists can make a preliminary screening and refer only pathological or doubtful cases to the ophthalmologists for a comprehensive examination. Therefore ophthalmologists’ workload can be reduced, and there is an increasing availability of specialist care when requested.

However, one finding raises some concern. There is a loss of chance for only the papillary cupping and the choroidal lesions with a significant higher number of missed diagnoses for the endocrinologists compared with the ophthalmology residents (P ≤ 0.05). Although these were not the main purpose of the training, this shows that training has to be improved for the associated findings.

In conclusion, DR screening by endocrinologists with three-field digital fundus camera and pupillary dilatation placed in a diabetic care unit is safe and secure, inducing no loss of chance for the patients when compared with identical evaluation performed by experienced ophthalmologists. This program allows us to attain the recommended objective of annual retina examination for all patients attending a diabetes department over 1 year. These results enable the delegation of DR screening to the endocrinologists using a digital camera. At a national health care framework level, delegation of DR screening to endocrinologists with no loss of chance could potentially increase the proportion of patients with diabetes who undergo this important preventive intervention.

This study was supported by a grant from the Regional Department of Clinical Research and Innovation (DRCI).

No potential conflicts of interest relevant to this article were reported.

N.G. performed the screening, researched and collected data, and wrote the manuscript. B.G. reviewed and edited the manuscript. N.D.-J. performed the screening and reviewed the manuscript. L.M. performed the screening and reviewed the manuscript. P.M. performed the screening. G.T. performed the screening and collected data. P.G. and B.E. supervised the study.

The authors thank Nathalie Vergely, Valérie Cerisier, Sandrine Lassandre, from the Department of Endocrinology, Diabetes and Metabolism, University Hospital, Saint-Etienne, France, and Anne-Frédérique Garcin, from the Department of Ophthalmology, University Hospital, Saint-Etienne, France, who also performed the screening. They also thank Dragos Galusca from Johns Hopkins Hospital, Baltimore, MD, for English editing. The authors are grateful to the regional DRCI for funding the hospital research project (Grant: Regional Clinical Hospital Research Trial 2003-28/04).