Implementation and Evaluation of a Low-Literacy Diabetes Education Computer Multimedia Application

Five urban outpatient clinics participated from June 2002 through October 2003. Each site received a personal computer with touch screen for patient access (installed to run continuously as a kiosk). Computers were placed in waiting areas with headphones available.

Medical assistants routinely asked patients if they had diabetes. During recruitment, patients with diabetes were invited to participate. Eligibility criteria included age 18 years or older, self-reported history of type 1 or type 2 diabetes, and verbal fluency in English or Spanish. Subjects were excluded if they were not directly involved with their own diabetes care. Among the 313 patients referred for participation, 11 were excluded because they never used the study computer, and 58 refused participation, leaving 244 study subjects (Fig. 1). Reasons cited for declining participation included lack of time, lack of interest in computer use, acute illness, and no need for additional diabetes education.

Before enrollment, 19 computer-based multimedia lessons were created in English and Spanish (Letterpress Software). Lesson content covered an introduction to diabetes, blood glucose management, oral medications and insulin, nutrition and physical activity, depression and stress, oral hygiene, and the prevention of complications (including eye, foot, cardiovascular, and kidney diseases). Users were recommended to begin with the introductory lesson; however, they could choose lessons in any order (visual cues indicated which lessons have been viewed). Each lesson targeted a specific self-care objective according to Gagné’s theory of learning and the component design theory (15,16). For each objective, a learning hierarchy was developed based on relevant verbal information (diabetes facts), intellectual skills (procedures for appropriate diabetes care), and cognitive strategies (suggestions for a healthy lifestyle). Each lesson incorporated Gagné’s events for instruction, including obtaining the user’s attention, stating the objective, reviewing previous learning, providing learning guidance, evaluating user performance, and providing feedback (17). The design included intervention tailoring (based on previous computer experience and learning style) as well as message tailoring of information (targeting communication of African Americans and Latinos) (18). Local professional talents provided instructional narration. To develop the software, a separate group of individuals with diabetes were video recorded for unscripted testimonials related to diabetes, emphasizing barriers to care, challenges, and personalized solutions they or family members have encountered. While the lesson plans for the English and Spanish versions were very similar through translation, different testimonials from various subjects were used to relate both language and culturally appropriate information to the users.

The program was available on touch-screen personal computers. Navigation was provided through a simplified interface, including forward/backward buttons for user control. Advanced features included “pop-up” supplementary text information or additional testimonials related to the concurrent screen concept. After each lesson, approximately 10 randomly sequenced multiple choice questions were presented for reinforcement. Individuals who incorrectly answered questions received immediate audio feedback. The average time for lesson completion ranged between 10 and 20 min.

A second multimedia application was developed for the control group. This group received simple multiple-choice quizzes on diabetes-related concepts. Unlike the intervention program, the control application did not include formal narrative instruction or testimonials.

Institutional review board approval was granted through the University of Illinois at Chicago. Surveys were performed via face-to-face interviews. Following completion of baseline weight, blood pressure, A1C measurements, and survey assessments, subjects were introduced to the computer. A bilingual research assistant provided initial instruction regarding usage, with input provided through the touch-screen monitor. Participants were randomly assigned by computer to one of two groups: the multimedia intervention group and the standard-of-care (control) group. The intervention group had access to the “Living Well with Diabetes” multimedia application, while the control group only had access to quizzes. Both groups were invited to use the computer as often as they would like during the following year and were encouraged to use it before or following visits with their providers. Subjects received compensation based on computer usage. Each day the computer was accessed, a subject received $5 per day (maximum $25 for five interactions over 1 year). Survey measurements and clinical data were repeated at 1 year. Upon study completion, a brief questionnaire was administered to obtain feedback regarding the subjects’ computer experience. Subjects were considered to have completed the study if they underwent the year-end interviews and used the computer at least once.

Subjects were offered the option of completing survey materials in English or Spanish. All surveys (excluding the literacy test) were conducted by face-to-face interview. Health literacy was evaluated through the S-TOFHLA (short form, 14-point font) literacy test in the subject’s preferred language (19). The literacy score minimum is 0 and maximum is 36. Survey data included adapted knowledge and self-reported medical care scales previously developed and validated using Rasch modeling (20). The medical care questionnaire included items based on the American Diabetes Association standards of medical care in diabetes (21). Subjects were asked if they underwent specialist evaluations (e.g., dilated eye exam), laboratory studies (e.g., A1C), and immunizations (influenza and pneumococcal vaccines) in the previous year (scored as “yes”, “no”, or “don’t know”). A 12-question self-efficacy scale was administered based on a similar scale developed for Spanish-speaking Latino populations (22) (adapted from the Insulin Management Diabetes Self-efficacy Scale [23]). Perceived susceptibility to complications was assessed by subjects evaluating their risk of diabetes complications on a scale from 1 to 10 (with 10 having the greatest risk) (24). Clinical data included fingerstick testing for A1C (A1C, Bayer DCA 2000 analyzer; range 2.5–14.0%). Weight, height, and blood pressure information were acquired from concurrent clinical visit data. BMI was computed as weight in kilograms divided by the height in meters squared. Study computers in the clinics recorded usage information for individual users, which was converted into time spent logged in and number of distinct computer sessions.

Analyses were performed on all complete cases using SPSS Version 11.0 (SPSS, Chicago, IL). Rasch models were created to determine person-measures of survey data (knowledge, self-efficacy, and medical care) using Winsteps (Chicago, IL) (25–27).

Health literacy scores were dichotomized to lower–and higher–health literacy subgroups, by combining inadequate (0–16) and marginal (17–22) scores together, as both groups share very limited computer experience. A similar threshold has also previously been applied (7,28). At baseline, we compared patients by group assignment and literacy subgroup using t tests or Mann-Whitney U tests for continuous variables and χ² or Fisher’s exact tests for categorical variables. A similar approach was used to evaluate factors related to attrition by comparing subjects who completed the study with those who dropped out.

The primary outcome was change in A1C over the 1-year study period. A repeated-measures general linear model analysis was performed. Analyses of the primary outcome included adjustment for baseline covariates if differences were found between groups at baseline (with P < 0.20 criterion). The following covariates were included in the model: age, sex, Latino race, income, insulin therapy, duration of disease, and previous attendance in diabetes class. Receiving information to read about diabetes and visiting a dietitian both correlated with diabetes class attendance and thus were not included in the model. Subject characteristics such as language spoken, insurance, clinical site, and highest education level attained were also associated with other included covariates. Results were very similar for unadjusted and adjusted analyses (the adjusted analyses are presented).

Similar analyses were conducted to evaluate intervention effects on changes in secondary clinical and survey outcomes. Group-by-time interactions were evaluated for all outcomes. A P value of <0.05 was considered statistically significant. To detect a 0.5% difference in A1C level at 1 year, a total sample size of 198 was necessary to retain a power of 0.8 with an α error of 0.05 (two tailed). The target sample size for enrollment was increased to 250 subjects, adjusting for anticipated attrition. The sample size was not powered to detect differences by literacy status.

Of 244 enrolled patients, 183 (75%) completed the 1-year trial. Subjects who dropped out had lower self-reported medical care measures at baseline (−0.51 vs. 0.029, P = 0.013) and were more likely uninsured (59 vs. 35%, P = 0.001). No differences were detected by baseline A1C, previous computer use by history, or literacy group.

Demographic and diabetes education–related variables are summarized in Table 1. Based on health literacy testing, 56% in the intervention group and 55% in the control group had low health literacy. The intervention and control groups were of similar lower socioeconomic status; however, the control group reported greater attendance at diabetes classes and dietitian visits, more subjects receiving educational information, and more subjects on insulin therapy. Lower-literacy subjects were more likely to be older, Latino, Spanish speaking, less educated, uninsured, and have lower income levels. They also were more likely to use insulin and less likely to own or ever have used a computer. While individuals with lower literacy had lower scores on diabetes knowledge assessments, there were no other significant differences in clinical characteristics at baseline.

Overall, there was no significant change in A1C, BMI, or blood pressure between intervention and control groups (Table 1). Among subjects with higher literacy, there was a small decrease in A1C in the control group (−0.5 vs. +0.3%, P = 0.043). Exploratory analysis of low-literacy subjects with poor glycemic control (baseline A1C ≥9.0% in 26 subjects) showed there was a greater decrease in A1C in the intervention group than control group (−2.1 vs. −0.3%, P = 0.036). For high-literacy subjects with poor glycemic control, there was no significant difference (−0.9% for intervention and −1.3% for control group, P = 0.548).

Overall changes in self-efficacy, knowledge, and medical care showed no differences for the intervention and control groups (Table 1). However, change in perceived susceptibility to complications was greater for intervention subjects, who reported an increased personal risk of having eye, kidney, or heart disease (1.19 vs. 0.24, P = 0.009). The greatest difference in perceived susceptibility was in lower–health literacy subjects (1.48 vs. 0.19, P = 0.016). While there was also change in the higher–health literacy group, this was not statistically significant (0.76 vs. 0.29 for intervention and control groups, P = 0.267). Among individuals with lower health literacy, there was a trend toward greater improvement in self-efficacy for the intervention group (1.51 vs. 0.99, P = 0.113). Among individuals with higher health literacy, there was no significant difference in self-efficacy gain. Medical care improved over time (P = 0.012 for time interaction) but without an intervention effect for either lower- or higher-literacy groups.

Recorded computer usage was analyzed for all 244 subjects. Mean time duration with the computer was greater for the intervention group (53.5 vs. 21.3 min, P = 0.001). The number of individual sessions during the study period was slightly greater for the intervention group but was not statistically different (2.9 vs. 2.3 sessions, P = 0.121). Within the intervention group, there was greater computer use among higher–health literacy participants (81.0 vs. 44.1 min, P = 0.006 and 4.0 vs. 2.1 sessions, P = 0.002). There was also a positive correlation between health literacy score and total time spent on computer (Spearman’s ρ = 0.284, P = 0.005).

Despite 74.3% of subjects describing an overall lack of experience with computers, 74.9% found the program easy to use (65.0% for lower-literacy vs. 87.5% for higher-literacy participants). Among participating clinical sites, total subject usage of the computer varied greatly (ranging from 6.5 to 32.6 h per location). For the intervention group, average computer time ranged from 19.5 to 116.7 min per subject per site. Several lifestyle changes were self-reportedly enhanced by computer use: diet, exercise, smoking cessation, cutting nails, and home glucose monitoring. Four common barriers to computer use were identified: lack of prompting by clinical staff, time constraints, availability of technology, and personal factors (e.g., difficulty in using the computer).

The computer-based multimedia intervention was successfully implemented in the clinical environment and resulted in a greater level of perceived susceptibility to diabetes-related complications but without improvement in glycemic control. There are several possible explanations for the latter negative finding. First, the intervention did not address provider factors such as intensification of therapy (or “clinical inertia”) (29,30). Second, the average A1C was relatively low at baseline (8.2%). The study did not incorporate a minimum A1C criterion in order to facilitate recruitment of a diverse population in a clinical setting and included individuals recently diagnosed with diabetes but unable to attend classes. In comparison, a similar approach using touch-screen computers in waiting rooms focused on dietary barriers had no effect on A1C or weight (31). Another randomized trial was conducted using tailored diabetes self-management training and support delivered via the Internet (14,32). This study demonstrated no significant change in A1C, though the intervention targeted exercise behavior and not glycemic control.

The intervention in the present study did result in significant improvement in A1C among low–health literacy subjects with poor glycemic control (A1C ≥9.0%). While this represents a fraction of the enrolled sample, it supports a potential indication for multimedia use, since few interventions specifically address low health literacy for patients with inadequately treated diabetes (33). Indeed, the multimedia users with lower health literacy demonstrated gains in knowledge, self-efficacy, and perceived susceptibility to complications compared with those having higher health literacy. Further research is necessary targeting individuals with poor glycemic control.

There are several mechanisms by which the multimedia may impact users. Prominent features of the educational application include video-recorded patient testimonials and interactive assessments that provide immediate feedback to users. Realistic role models may encourage vicarious learning through imitation. Also, skill building of glucose monitoring, insulin injection, and lifestyle modification are promoted. The use of video-recorded testimonials including persons with visual defects, kidney disease, amputations, and other complications of diabetes may elevate perceived susceptibility to complications among viewers. These testimonials highlight the benefits of self-care and discuss potential solutions to barriers. The additional use of positive feedback may increase self-efficacy via verbal persuasion through testimonial and narrator encouragement (34,35). The psychological impact of these features may be explained by various social and psychological health behavior change models, including Bandura’s Social Cognitive Theory and Rosenstock’s expanded Health Belief Model (36–38). In the setting of high perceived susceptibility to complications, behavioral change may occur if perceived benefits and barriers are subsequently addressed, particularly for those individuals having a greater sense of competence (or self-efficaciousness) to implement change (37).

The intervention also addresses the “digital divide.” This disparity not only represents differential access to computer technology and the Internet, but it also describes personal characteristics that separate users from nonusers of electronic information (39). In this study, while 34% of individuals with lower health literacy owned a computer, only 5% state that they had previously used a computer (compared with 49% of the higher–health literacy group). Additionally, 55% of tested subjects had inadequate or marginal health literacy, similar to other estimates of urban populations (3). Access to technology will not effectively enable dissemination of consumer health information if the content is unreadable. The developed multimedia addresses many of the needs and preferences of low-income and underserved Americans concerning content: information provided on a basic health literacy level, content for non-English speakers, and culturally defined content (40). Furthermore, the implementation of touch-screen monitors offers an alternative to keyboard and mouse for computer input, which may hold value for individuals who are older, have lower health literacy, or have less computer experience (41,42).

Based on our experience, there are several challenges to moving forward with computer-based interventions. Despite efforts made to improve computer usability, there was less use by lower–health literacy participants. As the technology was installed in the clinical setting, unexpected issues arose. The optimal location for computer accessibility was often uncertain. Making the computers visible, freely available, and accessible (meeting the guidelines of the Americans with Disabilities Act) was challenging in crowded waiting areas. Unfortunately, privacy is sacrificed when a computer is more centrally located within a clinic. Finally, the availability of computers may be limited for those waiting for others to finish use or when the computers malfunction.

Prompting, instruction, repetition, and support were key factors in computer use. Staff encouragement and initial assistance improved acceptance; however, changing staff behaviors in busy clinical environments is challenging. Bilingual research assistants provided brief initial instruction to users, which has been helpful particularly in locations lacking bilingual staff. Multimedia audiovisual prompting was found to be necessary for continued use. However, repeated sessions may require external prompting by staff, either through verbal reminders or provider recommendations for computer-based learning.

Strengths of this study include a randomized controlled study design carried out in a “real-world” setting. In addition, the intervention was evaluated in a population consisting of many individuals with low health literacy and who only speak Spanish. However, there were also several limitations to the study. Despite efforts in improving accessibility using touch-screen hardware and audiovisual media, there was less computer use among elderly and lower–health literacy participants, possibly dampening the potential effect of the intervention. More recent experience indicates much greater use and acceptance with a volunteer or family member helping to navigate the computer. Given that the study design involved five different clinical sites, various concurrent organizational-based and public health measures may have affected outcomes. Other studies of independent interventions (including the use of bilingual health educators and promoters/community lay workers) on minority groups are potential effect modifiers. Also, the study design was limited in its inability to include the collection of behaviorally relevant data, such as adherence to clinic visits or annual eye examinations. Thus, specific self-management improvements demonstrating behavioral change in response to specific multimedia lessons are difficult to estimate.

In conclusion, a new diabetes education multimedia application has been developed and successfully implemented in clinical waiting areas, targeting a vulnerable population. Individuals using the program had higher perceived susceptibility to complications but no change in glycemic control. Despite measures to improve access to individuals with lower health literacy, there was relatively less use of the computer among these participants. However, the intervention did result in significant improvement in A1C among low–health literacy subjects with poor glycemic control. Additional personal and organizational barriers must be addressed in conjunction with provider involvement to improve usability. This may increase the impact of computer-based education on behavioral and clinical outcomes.

Support was provided by the Agency for Healthcare Research and Quality (U18 HS11092). A.M.A. is supported by a research career development grant from the Health Services Research and Development Section of the Department of Veterans Affairs.

The authors wish to acknowledge Mariela Girotti, RN; Lourdes Pelaez, MD; Lori Brodsky, FNP, CDE; and Edgar Diaz for their contribution to multimedia development and study protocol.