OBJECTIVE

To evaluate the ability of certified retinal imagers to identify presence versus absence of sight-threatening diabetic retinopathy (stDR) (moderate nonproliferative diabetic retinopathy or worse or diabetic macular edema) at the time of retinal imaging in a telemedicine program.

RESEARCH DESIGN AND METHODS

Diabetic patients in a primary care setting or specialty diabetes clinic received Joslin Vision Network protocol retinal imaging as part of their care. Trained nonphysician imagers graded the presence versus absence of stDR at the time of imaging. These gradings were compared with masked gradings of certified readers.

RESULTS

Of 158 patients (316 eyes) imaged, all cases of stDR (42 eyes [13%]) were identified by the imagers at the time of imaging. Six eyes with mild nonproliferative diabetic retinopathy were graded by the imagers to have stDR (sensitivity 1.00, 95% CI 0.90–1.00; specificity 0.97, 0.94–0.99).

CONCLUSIONS

Appropriately trained imagers can accurately identify stDR at the time of imaging.

The American Telemedicine Association Telehealth Practice Recommendations for Diabetic Retinopathy identifies four categories of telemedicine care for diabetic retinopathy (1). Category 1 programs identify patients with no or minimal diabetic retinopathy (Early Treatment Diabetic Retinopathy Study [ETDRS] level 20 or below) versus those with diabetic retinopathy more severe than ETDRS level 20. Category 2 programs accurately determine whether sight-threatening diabetic retinopathy (stDR), as evidenced by any level of diabetic macular edema (DME), severe or worse levels of nonproliferative diabetic retinopathy (NPDR) (ETDRS level 53 or worse), or proliferative diabetic retinopathy (ETDRS level 61 or worse), is present or not present. Category 3 programs accurately identify ETDRS-defined levels of diabetic retinopathy and DME to determine appropriate follow-up and treatment. Category 4 programs can replace ETDRS 7-standard field 35-mm stereoscopic color fundus photographs in any clinical or research program.

The Joslin Vision Network (JVN) is a validated category 3 program (25). Imagers undergo an intensive 3-day program that includes fundus camera operation and imaging software navigation; structured courses on diabetes, ocular anatomy, diabetic retinopathy, and common ocular disorders; and a guided review demonstrating retinal images of nondiseased and diseased eyes. As part of the certification, imagers learn to recognize lesions of diabetic retinopathy, including hemorrhages, microaneurysms, venous caliber abnormalities, intraretinal microvascular abnormalities, retinal neovascularization, cotton wool spots, hard exudates, and laser scars. Salient retinal abnormalities not related to diabetes are also demonstrated, including choroidal nevi, retinal emboli, and large or asymmetrical optic cup-to-disc ratios. After the 3-day program, imagers serve a probationary period with senior imager supervision and ongoing quality improvement and assurance.

This prospective study assessed the ability of two certified imagers to conduct American Telemedicine Association Category 2 (presence vs. absence of stDR) grading at the time of retinal imaging.

Patients with diagnosed diabetes had nonmydriatic JVN imaging as part of their routine physical examinations in a primary care setting (HealthCare Associates, Beth-Israel Deaconess Medical Center) or a specialty diabetes clinic (Adult Diabetes, Joslin Diabetes Center). At the time of imaging, certified imagers (A.M.T., 71 patients [45%]; T.F., 87 patients [55%]) identified patients with potential stDR, defined for this program as ETDRS levels of 43 or worse (6) or DME, and ungradable images. Imagers were not able to manipulate the color, brightness, contrast, or other features of the images and could not view images stereoscopically. To grade retinal thickening without stereoscopic viewing, imagers relied on identifying hard exudates or microaneurysms within 3,000 microns from the center of the macula as surrogate markers for DME. The two certified imagers were Bachelor of Arts college graduates with no prior health care experience in evaluating retinal images and had not provided direct patient care before working as retinal imagers. Certified readers graded images according to the previously described JVN protocol (2,3) in a central reading center with calibrated monitors and stereoscopic viewing capability. All readers in the JVN program are Massachusetts-licensed optometrists. All readers were masked to the grading performed by the imagers. All findings were recorded on a specifically designed template.

A total of 158 consecutive patients were imaged. Mean age was 56.5 years (range 22–86), 54% female, and the mean diabetes duration was 7.0 years (range 0.1–42). A total of 316 eyes were evaluated, and 195 (61.7%) had no diabetic retinopathy, 62 (19.6%) had mild NPDR, 24 (7.6%) had moderate NPDR, 3 (1%) had severe or very severe NPDR, 2 (0.6%) had proliferative diabetic retinopathy, and 30 (9.5%) were ungradable for diabetic retinopathy. DME was absent in 266 (84.2%) eyes, present in 13 (4.1%), and 37 (11.7%) were ungradable for DME.

Of the 316 eyes assessed, imagers identified 48 (15%) eyes with potential stDR at the time of imaging. Subsequent grading by certified readers classified 6 (12.5%) of these eyes as mild NPDR. The imagers accurately identified all cases of stDR as graded by the readers. Although limited by the moderate sample size and the use of only two independent imagers, the agreement for determining stDR between imagers and readers was 0.95 ± 0.02. The sensitivity and specificity in identifying stDR at the time of imaging by a certified imager is 1.00 (95% CI 0.90–1.00) and 0.97 (95% CI 0.94–0.99), respectively (positive predictive value 0.88 [95% CI 0.74–0.95]; negative predictive value 1.00 [0.98–1.00]). There was complete agreement between imagers and readers regarding ungradable eyes (37 [12%]). Table 1 presents a cross-tabulation of imager and reader evaluations for the presence of stDR and ungradable images.

Table 1

Cross-tabulation of grading for the presence of stDR by JVN imagers and JVN readers

Cross-tabulation of grading for the presence of stDR by JVN imagers and JVN readers
Cross-tabulation of grading for the presence of stDR by JVN imagers and JVN readers

Film or digital retinal imaging is a sensitive method to identify the presence and level of diabetic retinopathy (710). Despite efforts to automate retinal image evaluation (1113), currently no system can perform such analyses in real time, and present methods of retinal imaging require trained imagers to acquire retinal images.

This study shows that appropriately educated and certified imagers following a clearly defined imaging and grading protocol can accurately evaluate retinal images with a high degree of sensitivity and specificity for the presence of stDR and inadequate image quality at the time of imaging. The ability to identify ungradable images and detect potential stDR facilitates reacquisition of retinal images during a single imaging encounter and allows prompt referral to appropriate eye care. Although this study involved a moderate number of eyes (n = 316), 42 (13%) eyes with stDR and 37 (12%) eyes with ungradable images were identified, representing all cases that would have required further ophthalmic evaluation and care. Additional studies with a variety of imagers and patient populations will be required to determine whether similar results can be obtained across diverse health care scenarios. However, the fact that the two certified imagers involved in this study had no prior health care experience in evaluating retinal images suggests that similar results are possible. In this study, retinal imagers had received a validated standardized method of certification and training, which is an important consideration when extrapolating these results to other retinal imaging programs.

No potential conflicts of interest relevant to this article were reported.

J.D.C. and P.S.S. researched data and wrote the manuscript. A.M.T., T.F., D.T., B.P., and S.E. researched data and reviewed and edited the manuscript. L.M.A. and L.P.A. reviewed and edited the manuscript and contributed to discussion. J.D.C. and P.S.S. are the guarantors of this work and, as such, had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

1.
Li
HK
,
Horton
M
,
Bursell
SE
,
Cavallerano
J
, et al
.
Telehealth practice recommendations for diabetic retinopathy, second edition
.
Telemed J E Health
2011
;
17
:
814
837
[PubMed]
2.
Ahmed
J
,
Ward
TP
,
Bursell
SE
,
Aiello
LM
,
Cavallerano
JD
,
Vigersky
RA
.
The sensitivity and specificity of nonmydriatic digital stereoscopic retinal imaging in detecting diabetic retinopathy
.
Diabetes Care
2006
;
29
:
2205
2209
[PubMed]
3.
Bursell
SE
,
Cavallerano
JD
,
Cavallerano
AA
, et al
;
Joslin Vision Network Research Team
.
Stereo nonmydriatic digital-video color retinal imaging compared with Early Treatment Diabetic Retinopathy Study seven standard field 35-mm stereo color photos for determining level of diabetic retinopathy
.
Ophthalmology
2001
;
108
:
572
585
[PubMed]
4.
Cavallerano
JD
,
Aiello
LP
,
Cavallerano
AA
, et al
;
Joslin Vision Network Clinical Team
.
Nonmydriatic digital imaging alternative for annual retinal examination in persons with previously documented no or mild diabetic retinopathy
.
Am J Ophthalmol
2005
;
140
:
667
673
[PubMed]
5.
Chow
SP
,
Aiello
LM
,
Cavallerano
JD
, et al
.
Comparison of nonmydriatic digital retinal imaging versus dilated ophthalmic examination for nondiabetic eye disease in persons with diabetes
.
Ophthalmology
2006
;
113
:
833
840
[PubMed]
6.
Early Treatment Diabetic Retinopathy Study Research Group
.
Grading diabetic retinopathy from stereoscopic color fundus photographs: an extension of the modified Airlie House classification. ETDRS report number 10
.
Ophthalmology
1991
;
98
(
Suppl.
):
786
806
[PubMed]
7.
Klein
R
,
Klein
BE
,
Neider
MW
,
Hubbard
LD
,
Meuer
SM
,
Brothers
RJ
.
Diabetic retinopathy as detected using ophthalmoscopy, a nonmydriatic camera and a standard fundus camera
.
Ophthalmology
1985
;
92
:
485
491
[PubMed]
8.
Li
HK
,
Danis
RP
,
Hubbard
LD
,
Florez-Arango
JF
,
Esquivel
A
,
Krupinski
EA
.
Comparability of digital photography with the ETDRS film protocol for evaluation of diabetic retinopathy severity
.
Invest Ophthalmol Vis Sci
2011
;
52
:
4717
4725
[PubMed]
9.
Hubbard
LD
,
Sun
W
,
Cleary
PA
, et al
;
Diabetes Control and Complications Trial/Epidemiology of Diabetes Interventions and Complications Study Research Group
.
Comparison of digital and film grading of diabetic retinopathy severity in the Diabetes Control and Complications Trial/Epidemiology of Diabetes Interventions and Complications Study
.
Arch Ophthalmol
2011
;
129
:
718
726
[PubMed]
10.
Gangaputra S, Almukhtar T, Glassman AR, et al. Comparison of film and digital fundus photographs in eyes of individuals with diabetes mellitus. Invest Ophthalmol Vis Sci 2011;52:6198–6173
11.
Chaum
E
,
Karnowski
TP
,
Govindasamy
VP
,
Abdelrahman
M
,
Tobin
KW
.
Automated diagnosis of retinopathy by content-based image retrieval
.
Retina
2008
;
28
:
1463
1477
[PubMed]
12.
Abràmoff
MD
,
Reinhardt
JM
,
Russell
SR
, et al
.
Automated early detection of diabetic retinopathy
.
Ophthalmology
2010
;
117
:
1147
1154
[PubMed]
13.
Fleming AD, Philip S, Goatman KA, Prescott GJ, Sharp PF, Olson JA. The evidence for automated grading in diabetic retinopathy screening. Curr Diabetes Rev 2011;7:246–252