Making a Difference With Interactive Technology: Considerations in Using and Evaluating Computerized Aids for Diabetes Self-Management Education

We begin by summarizing key findings from diabetes self-management–related research for each of the ITs examined (Table 1).

Self-monitoring of blood glucose (SMBG) is considered central to diabetes self-management. Hand-held SMBG devices automatically record time, date, and blood glucose level, as well as other data related to self-management (e.g., caloric intake, exercise level)6 and allow for transfer of data directly to health care professionals. In addition, ITs for SMBG can present feedback in a variety of ways, including average blood glucose for specified intervals and frequency distributions of blood glucose levels within preset ranges.

Piette and Glasgow7 reviewed five controlled studies testing the efficacy of SMBG ITs among type 2 diabetic patients. They found that only one8 (Rutten et al., summarized in Table 1) showed a decrease in HbA_1c levels compared with control conditions.

Other portable ITs that have shown promise for health assessment are Computer-Assisted-Survey-Interviewing (CASI) software. Use of a laptop computer for health assessments instead of a flesh-and-blood interviewer can obtain more reliable and valid responses from patients, especially on sensitive or stigmatized issues, including diabetes self-management.9,10 

With ITs for automated telephone disease management (ATDM), patients receive a call with predetermined frequency (e.g., weekly) from the automated system. A familiar voice (e.g., clinic nurse) offers patients opportunities to touch the telephone keypad in response to prompts to discuss self-care activities or hear self-care tips.

Piette et al.11 demonstrated that patients from a county health care system who had access to ATDM used it to report self-care information and accessed tailored self-management education. Those randomized to ATDM reported fewer depressive symptoms and higher self-efficacy for self-care compared to usual care. They also reported greater satisfaction with services received and improvement in health-related quality of life.11 

Use of messages tape-recorded by a familiar clinician in the ATDM system can help to bridge office visits with feedback and information for clients. They offer a promising alternative to costly staffing scenarios, particularly for publicly funded clinics.

In addition to being cost-effective, ATDM may be the ITs currently best able to help reduce health disparities related to racial/ethnic differences. Piette et al.11 demonstrated that the ATDM system works well with ethnically diverse patients. They also showed that Spanish speakers were more likely than English speakers to utilize the system for access to self-care tips.

CD-ROMs allow for the display of video and other large multimedia files and for very complex programming algorithms. Users of touch-screen CD-ROM ITs for diabetes self-management have found the technology easy to use. Glasgow et al.12,13 developed a CD-ROM program with touch-screen capability allowing patients to obtain immediate, personalized feedback on their barriers to healthy eating and to engage in diabetes self-management goal setting and problem-solving. Users of this CD-ROM and a revised program had greater improvement in dietary behavior13,14 and on serum cholesterol levels, but not on HbA_1c or health-related quality of life compared to patients randomized to usual care.

The Internet is only the latest IT available to address diabetes self-management, but it is perhaps the one with the greatest potential. The Internet adds a new dimension: interaction with other patients, both live (e.g., through “chat room” discussions) and sequentially (e.g., through threaded discussions or bulletin boards).

McKay et al.15 demonstrated the feasibility of using the Internet to offer electronic self-management support. Feil et al.16 demonstrated that, even among novice Internet users, the majority of primary care patients approached would participate in an Internet-based diabetes self-management program. A recent randomized trial demonstrated that having access to a World Wide Web site where one can participate in chat room discussions and post electronic messages related to self-management issues increased perceived support both on diabetes-specific and global support scales,17 but data on efficacy and long-term maintenance are not yet available.

Use of the Internet for health promotion extends beyond social support, as evidenced by the Comprehensive Health Enhancement Support System (CHESS) developed by Gustafson et al.18 for other chronic illnesses. Although CHESS is designed to promote quality of life, reduction of risk behaviors, and more efficient use of medical services among patients and caregivers with HIV, congestive heart disease, or breast cancer, its potential application to diabetes is clear. Gustafson et al. demonstrated in a randomized trial with HIV patients that CHESS, with treatment components including online support groups, interactions with health experts, and access to decision-making aids, increased social support, physical activity, and participation in health care.18 

ITs offer both important opportunities and some risks in interacting with electronic medical records, diabetes registries, and other types of electronic patient information. There is great potential benefit from designing appropriately tailored interventions based on electronic data containing a patient’s medical condition, medication regimen, and psychosocial and economic situations.

Such integration is, however, not likely in the near future because of complex issues involving systems compatibility, patient confidentiality, and proprietary and data ownership concerns. These issues are even more important and sensitive when considering the use of electronically recorded psychological assessment information to tailor treatments.

A number of optimistic claims and visions have been advanced relating to future applications of ITs to health care.19 However, as with most new and powerful technologies, there are also dangers associated with the unthinking use of ITs. Critics have raised concerns about negative impacts of ITs regarding potential depression and reduced social interactions associated with high levels of Internet use.20 There are also several ethical issues, including confidentiality, that are yet to be worked out.

It would be valuable to have a set of standards to apply to these evolving technologies to help distinguish actual advances in ITs from mere hype and ineffective or misguided applications. Two general documents provide a certain amount of guidance: the American Diabetes Association’s technical review on diabetes self-management education,21 which summarizes characteristics of effective diabetes self-management interactions, and the Science Panel on Interactive Health Communication guidelines,19 which specify information that IT programs should report. Summaries of key points in these recommendations are provided in Table 2.

As useful as these documents are, they do not address some issues specific to using ITs for diabetes (and possibly other chronic illness) self-management education. In addition, it would be helpful to have an easy-to-remember framework or limited set of questions that one could ask rather than needing to refer to the extensive set of issues in either the ADA or Science Panel reports mentioned above. We feel that two sets of issues are central: questions about behavior change principles and questions about outcomes. In the remaining sections, we briefly discuss a framework for evaluating each of these issues and then summarize what is known about the questions raised.

The essential elements of self-management training can be considered as a spiral of five interrelated activities (Figure 1).22,23 These activities are often applied in a clockwise order, beginning with assessment of and feedback on current self-management behaviors and related beliefs and progressing to collaborative setting of specific self-management goals, identification of barriers to and social environmental supports for accomplishing these goals, development of individually tailored problem-solving strategies for overcoming obstacles, and one or more modalities of follow-up support.

All of these activities lead to the development and refinement of a personal action plan, as shown in the center of Figure 1. It should be emphasized that self-management training is not a one-time experience; rather, it should be an ongoing and regularly updated part of patient care.

Table 3 summarizes the strengths and weaknesses of ITs on each of these action steps. Administration, immediate scoring, and feedback on assessment instruments are clear strengths of ITs. Built-in error traps to minimize missing and out-of-range values and skip patterns to omit items not relevant to a given individual are standard. Feedback can be provided in a variety of engaging formats. The primary disadvantage of most current IT programs is limited ability to explore assessment issues in an open-ended manner.

ITs do a reasonable, but not stellar, job of helping patients set goals. ITs are strong on generating a list of potential options for patients to choose from, but are generally limited in their ability to arrive at collaborative decisions. Until reliable voice-recognition technologies become standard, it can also be awkward to enter patient-generated goals into IT programs.

Identification of barriers and potential support resources is another strength of ITs. Computers can quickly administer and score assessments tailored to patients’ selected goal areas. As noted above, ITs also excel at providing feedback in a variety of formats.

In contrast, the problem-solving abilities of most current ITs are only moderate. Although ITs can reliably generate a list of theoretically relevant problem-solving strategies, they are limited by the range of goal/barrier pairings that have been preprogrammed. It is also challenging to convey a sense of true collaboration, rather than leaving patients feeling that they are “being told what to do by the computer.” One advantage of ITs, especially CD-ROMs, is that realistic video depictions of problem situations can be incorporated.

ITs are fairly strong on follow-up support. A variety of computer-generated follow-up modalities are available, and ITs are very persistent at making contacts. Many ITs also have the advantage of being available on demand, 24 h a day and when patients may require follow-up support. Finally, the generation of personal action plans is a strength of ITs. Plans can be quickly generated, stored, and transmitted to multiple sources in a variety of ways.

In summary, current ITs offer advantages of speed, availability, consistency, and tailoring in performing routine tasks and activities that have been programmed. They are, however, more limited in dealing with novel situations that have not been anticipated.

The RE-AIM framework4,5 was developed to more thoroughly understand the impact of health interventions. It emphasizes that five components are critical for assessing overall impact: Reach, Efficacy, Adoption, Implementation, and Maintenance. Consideration of how many people will participate in an intervention (Reach), how many organizations Adopt the program, the degree to which an intervention is Implemented as intended, and how long the effects of an intervention can be Maintained, are all important to consider in addition to a program’s Efficacy. RE-AIM was developed to place an equal emphasis on internal and external validity (generalization) issues and thereby to facilitate the translation of research into practice.

Table 1 considers the impact of various ITs using the RE-AIM framework. With a 60–93% enrollment rate in the studies reporting such data in Table 1, it appears that ITs have the potential to reach a high proportion of diabetic patients.

There are, however, critical concerns that a “digital divide” exists, resulting in differential access to ITs by traditionally marginalized racial and ethnic groups and people with lower education or income levels.24 Some recent studies show that this divide is shrinking among older adults and women, as well as in some ethnic groups.25 

Glasgow and Toobert13 and Feil et al.16 have demonstrated that older age and lack of computer familiarity are not barriers to access and use of diabetes-related ITs. Reach varies with IT modality (handheld devices, telephone systems, CD-ROMs, or the Internet). At present, ATDM has the greatest reach, but other modalities are closing this gap.

ITs have been demonstrated to be moderately efficacious, with the largest effects on behavioral rather than biological or quality-of-life outcomes. Piette and colleagues7,11,26 have shown that ATDM can affect depression, self-efficacy, and patient satisfaction. CD-ROMs and the Internet show efficacy on behavioral, some biological, and quality of life outcomes.

Adoption of ITs at the health system (e.g., organizational) level has been understudied. Most of the studies shown in Table 1 do not discuss actual diffusion but rather indicate how many settings were willing to participate in a research project. Where it is discussed, adoption appears high—90% with both Internet17 and CD-ROM13 studies.

Broader adoption of ITs can accomplish two goals: to ensure better access to ITs by people who use these organizations and to lay a foundation for maintenance.18 ATDM has been used with success in a county veterans’ hospital, although final adoption data have not been reported.11 As providers realize the efficacy of ITs and their potential for reducing staff time, ITs may be more readily adopted.

ITs have implementation advantages over other interventions.5 The ability to preprogram algorithms that are delivered consistently is a strength of ITs.

There are challenges, however. Barriers to implementation include installation and maintenance of hardware, implementation and complexity of some programming algorithms, and the necessity to transfer among different programs and databases in multi-session IT interventions. The only study in Table 1 that reported implementation showed a rate of 80–100%.13 

Maintenance questions arise at both the individual and setting levels:

1.  Will people continue to use and benefit from ITs in the long run? Assessing maintenance of IT interventions may be premature, given the short time we have studied their efficacy. Only one of the studies in Table 1 presents data for even 1 year postintervention.12 These authors also demonstrate the cost-effectiveness of the intervention, which may be critical not only for maintenance, but also for broader adoption. The ATDM studies show consistent use of the telephone interventions over time, demonstrating the potential for long-term maintenance.

2.  Can organizations maintain IT interventions and institutionalize their use? This question cannot be answered from available data; ITs have not been used for a long enough period of time to judge. However, the proliferation of ITs and the increasing use of electronic medical records suggest the potential for system-level maintenance.

The sections above provide justification for both optimism and caution regarding the contributions of ITs to diabetes self-management. Few data are currently available on most of the key issues concerning the reach, implementation, and long-term effects of ITs in diabetes or on optimal conditions of IT use. Therefore, the discussions above are based largely on extrapolations from the more general IT literature, on our own experiences, and on deductions and questions from diabetic patient education research.

Figure 2 compares in-person counseling to ITs on all dimensions of the RE-AIM framework. While in-person counseling has strong efficacy and is widely adopted, ITs show potential for greater reach and more consistent implementation. Figure 2 illustrates the potential of ITs for providing complementary efforts to promote diabetes self-management. As health care costs continue to be of concern and people seek more convenient access to care and information, ITs can offer appropriate adjunctive support to in-person education.

We conclude that ITs do not currently appear to be sufficiently sophisticated or data-based to be recommended as the sole modality for diabetes self-management education. Rather, their optimal use is as a supplement to other forms of patient education. As summarized above, ITs do have some advantages over human interactions and should be able to produce greater efficiencies in care to allow humans to focus their efforts on the things they do better than computers. The intent of ITs should not be to replace health care professionals, but rather to inform both patients and educators so that their interactions can be more productive.27 

Use of ITs must be accompanied by strong protection of confidentiality. Steps should include encrypting e-mail contact, advising participants if contacts are not encrypted, securing entry to Websites,28 informing participants of how data will be used, and keeping data in secure files with limited access.

Concerns have been raised about the multitude of health sites on the Internet and about the quality of information offered through these sites.29 Formal efforts are underway to consider ethical issues related to IT-related research and to assess the quality of ITs, particularly Websites.30 

Clearly, more research into diabetes ITs is needed, especially in the following areas:

1.  Using RE-AIM or similar frameworks to evaluate more comprehensively both the benefits and potential harms resulting from ITs.

2.  Determining the best ways to link IT-mediated diabetes self-management with other aspects of patient care and to integrate ITs into primary care.

3.  Research on the role of peer support in ITs and the use of ITs to promote self-management between visits with health care providers and to enhance long-term maintenance of behavior change.

The information age and ITs are here to stay. Our responsibility as health care professionals is to see that these new tools are used in ways that enhance rather than interfere with patient autonomy and self-efficacy and to carefully evaluate the impacts of such use. Until more evidence-based guidelines are available, practitioners are advised to 1) evaluate intended IT uses against both the five self-management support activities in Figure 1 and the ADA guidelines for diabetes self-management21 and 2) refer to the guidelines from the Science Panel on IT19,29 and the RE-AIM dimensions when developing or selecting IT applications.4,5 

Preparation of this article was supported by grant #039756 from the Robert Wood Johnson Foundation, grant #35524 from the National Institute of Diabetes and Digestive and Kidney Diseases, and grant HS 10123 from the Agency for Health Care Research and Quality. Appreciation is expressed to Dr. Richard Rubin for his helpful comments and editorial suggestions on an earlier version.