What Is the Sensitivity and Specificity of Self-Report for Retinopathy Screening?

A wide range in screening rates for diabetic retinopathy by dilated ophthalmic examination has been reported in studies over the last decade (1–4). Comparisons of these rates should take into account the measure used to assess the condition (i.e., screened versus unscreened). Examples of measures include self-report by the individual, claims data for the service, or medical record abstraction of the desired information. Because goals for improvement of diabetes care (5) are based on studies using these reported rates, the sources of these data must be considered and described. Self-report by an individual relies on memory, correct discernment of a dilated versus undilated exam, and accurate identification of the screening behavior within the time frame in question. Claims data, on the other hand, can overestimate the rate of dilated exams under the code for a comprehensive ophthalmic examination. Furthermore, depending on the patient’s health care coverage plan, the retinopathy screening may be a service that does not require a separate claim. Finally, abstraction of medical records may be a sensitive measure for a dilated ophthalmic examination; however, its specificity is dependent on a record system that captures and organizes the medical reports for retrieval and evaluation.

A recently published study by Basch et al. (6) reported that a telephone-based health education intervention was associated with a doubling in the rate of dilated ophthalmic examinations (54.7% screened in the intervention group vs. 27.3% in the standard care group) in a sample of African-American adults with diabetes. The main outcome for that randomized controlled trial was documentation from the medical record of receipt of a dilated eye examination. In addition, staff conducted telephone interviews to ascertain self-report of a dilated eye examination. The cross-classification of documented and self-reported screening status from that study is presented here in order to describe the accuracy (7) of self-report of a dilated eye examination.

Eligibility for the randomized trial specified that participants had not had a dilated eye examination in the previous 14 months. The current standard of diabetes care is for an annual dilated eye examination (8). A total of 280 African-Americans with diabetes from five New York City metropolitan area medical centers consented to be randomized and completed the study. Documentation of a dilated eye examination consisted of medical record abstraction by an auditor masked to group assignment. If in the telephone interview the subject reported having gone to an outside eye care provider and signed a medical release, then a form for provider documentation of the date of the most recent dilated eye examination was obtained. Because the documented outcome from the medical record would have superceded the patients’ self-report as the primary outcome for this study, completion of the telephone interview was not a prerequisite for inclusion in the published report. There was no self-report for 32 subjects because of the staff’s inability to contact the subjects by telephone, even after repeated attempts. Each of the remaining 248 subjects included in this current report had both self-report and accessible medical record documentation for eye examination status. Inaccessible medical records for seven subjects (three intervention subjects and four control subjects) were conservatively treated as a negative for the dichotomous categories of screened versus unscreened.

The telephone interviewer, masked to group assignment, asked, “When was the last time you had an eye exam in which the pupils were dilated?” Response options were: “less than 1 month,” “1–12 months,” “13–24 months,” “more than 2 years,” and “never.” We allowed plus or minus 1 month for error in recall. Thus, if a person reported having an exam in the past 1–12 months, and documentation indicated an exam 13 months ago, the self-report was considered verified. Similarly, the self-report was allowed to underestimate the time since last exam.

Rates of self-reported and documented dilated eye examinations for the entire sample (n = 248) and for the intervention (n = 119) and control (n = 129) groups separately are as follows: self-report was 58.5% for total sample, with 75.6 vs. 42.6% for intervention and control groups, respectively; documentation of exam was 44.0% for total sample, with 60.5 and 28.7%, respectively, for intervention and control groups. The apparent overestimation due to self-report was 33% in the total sample (58.5 vs. 44%), 48% in the control group (42.6 vs. 28.7%), and 25% in the intervention group (75.6 vs. 60.5%). The sensitivity of self-report for the total sample was 94.5, with intervention subjects showing greater sensitivity in self-report. Specificity overall was 69.8, with intervention subjects at 55.3 and control subjects at 77.2.

Objective 5-13 of Healthy People 2010 (5) is to increase the proportion of adults with diabetes who have an annual dilated eye examination from a 2000 baseline of 47% to a goal of 75%. The baseline was derived from self-report data. The National Health Interview Survey (NHIS) rate for African-Americans was 43%, which is virtually the same as the self-report data for the sample of African-Americans assigned to the control group, where the overestimation compared with documentation was by 48% (5).

These data indicate that self-report of a dilated ophthalmic exam and the documentation from the medical record or provider confirmatory letter are not in complete agreement. In fact, the intervention group had much less difference between the two measures than the control group. This difference could be because of a greater awareness of what constitutes a dilated eye examination, a result of the telephone-based health education and the brief behavioral counseling to improve retinopathy screening rates. Thus, accuracy of self-report data may be improved with greater patient education about what constitutes a dilated eye examination.

Ascertaining the true condition of screened versus unscreened for each subject was beyond the scope of this study and is a limitation of this report. It is possible that a subject’s self-report of having had a dilated eye examination was indeed factual, but medical records were misfiled or incomplete; the great majority (>90%) of subjects in this report, however, received eye care from medical centers using a shared medical record for individuals among clinics. There is not an alternative gold standard measure for receipt of a dilated eye examination, absent the observation of the event. However, more advanced databases and electronic medical records should improve the accuracy of documentation in the near future, and this, in turn, will improve the benchmarking for retinopathy screening as a standard of diabetes care.