Randomized Effectiveness Trial of a Computer-Assisted Intervention to Improve Diabetes Care

The Diabetes Priority Program was a collaboration between our research team and the Copic Insurance Company, which provides malpractice insurance to >95% of the independent primary care physicians in Colorado. Details of physician and patient recruitment have been reported previously (16,17) and are summarized herein. An initial survey was sent to all 1,258 family physicians and general internists insured by Copic Insurance Company in Colorado; 1,059 (84%) returned a useable survey and received a project fact sheet and follow-up letter inviting their participation. After a physician agreed to participate, a standard protocol was used to generate lists of patients with diabetes, detailing review of patient billing data for the previous year, diagnostic codes, and the need to search all diagnoses for each visit.

All adult patients identified as having diabetes were sent a letter signed by their primary care physician inviting them to participate, a brochure describing the project, and an opt-out postcard to return if they did not want to participate (Fig. 1). To make the project as broadly applicable as possible, the only inclusion criteria were age ≥25 years, ability to read English, and type 2 diabetes, confirmed using the Welborn criteria (18). All procedures were approved by relevant Institutional Review Boards, and patients were recruited during 2001–2002.

We used a two-group, cluster, randomized design. Participating physicians were stratified by size of practice and urban/rural setting, because these factors were judged likely to impact results, and then randomized. To avoid contamination, all physicians within a given clinic were assigned to the same condition. Randomization was conducted by the project statistician, who then notified research staff of condition assignment. To account for clustering of patients within physician, which could induce a correlation among the responses due to patterns of treatment associated with the clinician, a generalized regression model using a random effect for the physician (a mixed model) was fitted using the methods of Laird and Ware (19), adjusting for baseline score on the dependent variable. A required sample size of 32 physicians and 774 patients was determined using calculations to have 90% power (α = 0.05, two tailed) to detect a moderate effect, assuming an intraclass correlation as large as 0.05, and allowing for 20% attrition.

Differences between participants and nonparticipants, and between conditions at baseline, were conducted using mixed model linear regression analyses. Outcomes were evaluated using mixed model regression analyses (to account for clustering) and controlling for baseline scores on the dependent variable and any other potential confounding variables.

Participants assigned to the Diabetes Priority Program were asked to come 30 min early to two diabetes-related visits, scheduled ∼6 months apart, to complete the computerized touch screen assessment and action-planning procedure. The first part of the interactive computer program focused on the medical care participants were receiving for diabetes. Participants were asked to recall when they had last received each of the 11 items contained in the PRP measures (14) (http://www.ncqa.org). Responses were multiple choice, using 6-month intervals (e.g., within last 6 months, 7–12 months ago). Seven of these items involved procedures performed or ordered by the physician (e.g., blood pressure check, measurement of cholesterol levels, foot examination, measurement of microalbumin levels, dilated eye examination). Lipids and HbA_1c assessments were performed in all participants as outcome measures and thus were not eligible for inclusion as care process measures. The remaining five laboratory assessment procedures shown in Table 1 were summarized to produce a summary score of number of laboratory assessments meeting PRP criteria (15). The four “patient-centered” items that involved counseling for the patient on lifestyle aspects of the PRP measures (i.e., setting a self-management goal, receiving nutrition education or therapy, self-monitoring of blood glucose, and meeting patient satisfaction items) were summarized into a patient-centered composite (Table 1).

The second part of the computerized program focused on development of a self-management action plan. Patients answered questions on their dietary, physical activity, and smoking behaviors and received feedback on each of these. They next selected a behavior change goal in the area of smoking, diet, or exercise. The program guided participants through an interactive session that included selecting specific activities to support the goal area they chose, identifying barriers, and choosing strategies to help them overcome these barriers.

Three printouts were generated: an action plan for the patient, which included a summary of assays/checks the patient might be due for and a copy of the self-management plan; a summary of the patient’s needed assessments and self-management goals for the physician, including prominent notation of areas the patient wished to discuss; and a detailed printout to be used by the office’s designated “care manager.” The care manager was an existing clinic staff member (usually a nurse or medical assistant) who conducted a brief counseling session with the patient. Care managers were trained to use a patient-centered self-management approach (9,20) that included review of the medical care needs and self-care goals that the patient identified and brainstorming additional strategies that patients could use to overcome barriers to their goals. This took an average of 8–10 min. The care manager also attempted a brief follow-up call after each visit to review progress and to reinforce strategies developed. These procedures were repeated at the patient’s next visit (at ∼6 months). The intervention was designed to be consistent with recommendations from the Chronic Care Model for self-management support (2,21,22) but feasible to implement during primary care visits (17).

Patients of physicians in the control condition completed a touch screen computer assessment procedure involving the PRP measures described above, as well as general health risk appraisal items (e.g., use of seatbelts, cancer screening). They had the same number of visits as intervention patients and received a printout, but one that focused on general health risks and risk reduction that did not address the PRP measures. They did not meet with or receive calls from a care manager.

The 52 physicians had different medical record formats, very few had an established diabetes registry, and almost none reliably recorded most of these PRP activities. Therefore, we used patient reports of having received these services as our primary outcome measure. The scales described above were used in two previous studies with very similar patients (3,22) and were found to be reliable and to agree well with medical records in a health care system that had an electronic diabetes registry. Secondary outcomes included the following: 1)The revised Problem Areas in Diabetes 2 (PAID-2) scale, a revision of the original scale, assessed diabetes-specific quality of life (23). The earlier version has been demonstrated to be reliable and sensitive to change (23,24). In this study, the PAID-2 had an internal consistency of α = 0.93. 2)HbA_1c assays were conducted at the University of Colorado Health Sciences Center using a National Glycohemoglobin Standardization Program certified Bio-Rad Variant 2 analyzer (Bio-Rad, Richmond, CA), correlated to an index of glycemic control established during the Diabetes Control and Complications Trial. Its reference range was 4.1–6.5%. 3)Total cholesterol was assayed using an enzymatic test with high-performance liquid chromatography methods, achieved by microbial esterase, to ensure virtually complete hydrolysis (>99.5%) of all cholesterol esters. This process allows for direct comparability to Centers for Disease Control and Prevention (CDC) reference procedures. HDL cholesterol was assayed using the Roche direct HDL cholesterol automated method (Roche, Montclair, NJ), which meets the National Institutes of Health/National Cholesterol Education Program goals for acceptable performance. To avoid colinearity and to reduce the number of dependent variables, we used the ratio of total to HDL cholesterol as our lipid outcome.

4)The Patient Health Questionnaire (PHQ) is a self-administered instrument that has been validated as a depression diagnostic and severity measure (25). The PHQ-9 scores each of the nine Diagnostic and Statistical Manual, 4th edition depression criteria on a scale of 0 (not at all) to 3 (nearly every day). A score of 10 has been documented to have a sensitivity of 88% and a specificity of 88% for major depression (25). In the present study, the scale exhibited good internal consistency (α = 0.86).

A total of 52 physicians participated, consisting of 22 internal medicine and 30 family practice physicians from 30 clinics throughout Colorado. Based on an initial survey of physician characteristics, participating physicians did not differ from the total sample of 1,059 primary care physicians insured by Copic on age or sex of physician, years in practice, size of practice, or use of any of a series of several quality improvement processes for diabetes. Characteristics of participating physicians did not differ between conditions (Table 2). The number of patients participating per practice ranged from 13 to 61 (median 28).

Participants’ characteristics matched those of a random sample of Colorado diabetic patients, as detailed elsewhere by Glasgow et al. (17). Initial analyses failed to show baseline differences between conditions on any sociodemographic or baseline measures (Table 2).

Attrition rates were approximately equivalent (19% in intervention and 15% in control), modest across the two conditions at the 12-month follow-up (Fig. 1), and not due to any consistent reasons. There were no differences between the two conditions in the characteristics of patients who dropped out. Therefore, analyses were conducted on complete cases. Analyses using intent-to-treat procedures (and assuming those lost to follow-up at 12 months were performing at their most recently collected levels) produced identical conclusions.

Patients were receiving high levels of care at baseline, especially for the laboratory assay measures (Table 1). A total of 58–99% of patients were already receiving recommended services, which is substantially more than in two previous studies of similar samples using this measure (3,21). Despite this high initial level of care, patients in intervention practices showed significantly greater improvement on both laboratory assays (F = 11.6, P < 0.001) and patient-centered (F = 39.5, P < 0.001) subsets of the PRP measures.

Subsequent to overall significant effects, analyses were conducted using only those patients for each measure that did not meet National Committee for Quality Assurance/American Diabetes Association–recommended levels at baseline to account for potential ceiling effects (e.g., only those not reporting having undergone dilated eye examination within the past year at baseline). On three of the four measures for which there were more than 100 patients, the intervention condition produced superior results on the percentage of patients meeting criteria at 12 months (P < 0.05). All of the PRP measures having 30 or more patients showed trends favoring the intervention condition. These patients received rates of care averaging 17% higher than control subjects (range 5–40% higher). The greatest differences in improvement between conditions were on medical nutrition therapy, self-management goal setting, dilated eye examination, and foot examinations. These were also the areas in which performance was lowest at baseline.

Both conditions improved on measures of lipids, HbA_1c, quality of life, and depressive symptoms, but there was not a significant difference between conditions (Table 1). On HbA_1c levels, patients displayed relatively good levels of baseline control (mean = 7.3%). Therefore, subanalyses were conducted only on those patients exceeding HbA_1c of 8.0 at baseline and produced similar results.

Patients demonstrated a moderate level of baseline mood disturbance; 16–19% of patients scored at levels on the PHQ-9 that would be considered clinical depression. Both conditions improved significantly over time on these outcomes, but between-condition improvements were not differential.

Intervention patients only also established goals and developed action plans for behavior change in the areas of healthy eating, physical activity, or smoking cessation. Repeated-measures ANOVA to evaluate change showed that participants generally were successful in making improvements in these challenging lifestyle areas. Significant (P < 0.002) and meaningful improvements were seen in all areas except for smoking cessation. A total of 12% of the 35 baseline smokers who set cessation goals quit, but the sample size precluded significance.

The protocol was consistently implemented across the heterogeneous settings. At the 6-month intervention visit, 93% of patients received the computer-based interactive assessment procedure, 73% discussed the printout with the physician, 77% met with the care manager to discuss lifestyle goals, and 67% received at least one follow-up phone call. These results are lower than at the initial intervention session (99, 92, 99, and 86%, respectively) but still good for an effectiveness study implemented by clinical staff with many competing demands.

There are few “practical clinical trials” upon which to base clinical and policy decisions (8–10). We included several components to make this study more generalizable than the typical efficacy study. The intervention was delivered by regular clinical staff in a variety of settings, and few exclusion criteria were used (e.g., patients having comorbid conditions including depression were included); the program was conducted during usual medical care visits rather than special research appointments. Also, the touch screen computer was designed to be user friendly and usable by low-literacy patients (questions and information were presented aloud as well as on the screen), and the intervention was designed to fit into the flow of usual care. These actions were generally successful in making the intervention practical yet effective. Our assessment plan also included multiple outcome measures, including those important to clinicians, patients, and policy makers (10,26).

Overall, intervention effectiveness was moderate compared with the stringent control condition. The magnitude of effect was likely attenuated by good baseline levels of care and glycemic control, by limited variability on some measures, and because intervention was delivered by clinical staff. Still, the intervention was successful in increasing both care procedures and patient-centered lifestyle counseling—especially on measures in which there was the greatest room for improvement. These improvements were seen across different measures across a variety of primary care practices and types of patients. Some of the improvement in quality of care observed may have been due to patients who were new to a practice, since at baseline they would not have had a full year to receive the various services. We do not have information on how long patients had been with their provider, but this should have affected both conditions equally.

As reported in several other trials, these care process improvements did not translate into significant effects on biologic or quality-of-life improvements over the time period studied (27–29). Although the program did not enhance quality of life or reduce depression levels more than the control condition, both conditions showed improvement on these outcomes. Intervention patients and providers were dealing with more regimen, lifestyle, and guidelines issues without a reduction in quality of life or other apparent adverse consequences. Some of the improvement in both conditions may have been due to the increased attention to the care issues assessed.

The finding that reported improvements in care processes did not translate into improved outcomes has alternative explanations. The linkage between recommended care activities and outcomes may not be strong enough to override numerous potential confounding variables in real-world research, given the limited person-years of observation in this study. Alternatively, it may be that baseline levels of outcome measures were sufficiently good that it was not possible to detect improvement (e.g., to be significant on HbA_1c, intervention would have had to reduce average levels <7.0).

Regular primary care office staff delivered the intervention consistently despite competing demands. Almost all patients received the touch screen computer intervention, and even more staff-intensive aspects of the protocol such as meeting with a care manager and follow-up calls were completed at relatively high levels for an effectiveness study. We think this was due to the specificity the protocol provided, the integration of the computer into usual care, and the operational support our staff provided. We conclude that it is feasible to deliver an intervention such as the Diabetes Priority Program in small, independent, mixed-payer primary care practices.

This study has both methodological strengths and weaknesses as well as implications for practice, policy, and future research (8,10). Strengths include the scope of the study, the cluster randomized and appropriately analyzed design (30), the stringent control condition that also received computer assessment and feedback, the breadth of outcomes and patient-centered measures included, analyses that evaluated potential for translation, and the “practical clinical trial” focus (8,10). Limitations include the absence of “gold standard” registry or electronic medical records data (very few of the practices had diabetes registries). Recent reports have documented that nonelectronic medical records in most primary care clinics do not routinely document information on recommended preventive services and that patient self-report of diabetes care received is generally accurate (3,22,31).

Future research is needed to investigate characteristics of medical practices that are associated with outcomes, including reach, effectiveness, implementation, and maintenance (8,32). Interactive computer technology will be used increasingly in future research and practice (12,33). It is possible that greater benefit could be obtained through using interactive technology to address lifestyle change issues such as physical activity and nutrition counseling that are not often dealt with sufficiently in primary care (11,12,33–35).

This work was supported by the Agency for Health, Research and Quality, grant HS-10123.

We thank the multimedia teams at Klein Buendle and InterVision Media; the collaboration of Copic Insurance Company and our collaborating primary care partners without whom this research would not have been possible; Physician Recruiter Cecelia Holland, Patient Recruiter Roxane Smith, Biostatistician Monika Baier, and Data Manager Wendy Gehring, of the Cooper Institute; and Barbara McCray for her ongoing administrative assistance.