A Controlled Trial of Web-Based Diabetes Disease Management : The MGH Diabetes Primary Care Improvement Project

Study providers practiced in a hospital-based staff-resident practice, the Adult Medicine Clinic (AMC), where 39 staff physicians are divided into three groups. There is little organized interaction between providers in different groups, minimizing potential contamination in group-specific interventions. Staff and residents provide continuity of care for their own patient panels. Residents (and their patients) were members of the same group as their staff preceptor. Staff members were systematically exposed to the intervention, receiving formal preintervention training and two interactive feedback sessions during the intervention year. Residents were made aware of the DMA during an ambulatory training lecture on diabetes care. The AMC uses paper-based medical records and an electronic medical record (EMR; a web-based version of COSTAR [Massachusetts General Hospital, Boston, MA]) (25); computers with web browsers were available in every patient care room. The study was approved by the MGH/Partners Institutional Review Board.

The study was a group randomized, controlled trial (26); its design is shown in Fig. 1. A coin was tossed to select an intervention group and a control group. The intervention was conducted from May 1998 through April 1999. At the end of the intervention year, patients with at least one visit to the AMC during the preintervention year (May 1997 through April 1998) were identified by billing claims, and patients with type 2 diabetes were identified by ICD-9 codes 250.00–250.90. Of 1,595 patients with diabetes, 997 were excluded, as shown in Fig. 1; 598 patients were included in the study, and their charts were fully abstracted.

The centerpiece of the intervention was a web-based information management/clinical decision support tool, the diabetes DMA. The DMA (Fig. 2) provides a single-screen view of patient-specific information, enabling decision support at the time of patient contact. It displays trended and tabular real-time electronic laboratory data interactively linked to evidence-based treatment recommendations, facilities to aid encounter workflow, and links to additional patient and provider care resources. Treatment recommendations were based on our synthesis of published efficacy data and national guidelines current at the time of the study (9–11). A guideline engine generating patient-specific recommendations used GuideLine Interchange Format (GLIF); data were passed to and from the guideline engine, HTML tables, and a Java graphing applet in XML (eXtensible Markup Language) format (27,28). The DMA is not an involuntary reminder or prompting system: it must be actively opened, just as any on-line reference or EMR application. During the intervention year, the DMA was available to intervention providers to use if they wished. Control providers continued their usual care practices during the intervention and did not have access to the DMA.

Clinical data from paper and electronic charts were abstracted by three nurses blinded to group status of providers and patients. Data were abstracted into an electronic database featuring explicit item criteria displayed adjacent to the data entry field. To assess interabstractor variability, all nurses abstracted the same 42 randomly selected charts. κ Statistics or intracorrelation coefficients (ICCs) indicated good to excellent agreement for all items: κ during the preintervention year and the intervention year, respectively, were 0.61 and 0.62 for eye examinations, 0.43 and 0.55 for foot examinations, 0.42 and 0.76 for microvascular complications, and 0.68 and 0.72 for CVD complications at baseline; ICCs were 0.94 and 0.81 for systolic blood pressure and 0.96 and 0.93 for diastolic blood pressure during each year. Laboratory data were obtained from hospital electronic databases. We also collected demographic and practice information about staff providers and administered a questionnaire (response rate 67%) to assess their use of practice guidelines.

Clinical characteristics were determined from documents such as problem lists and hospital discharge summaries. We defined characteristic or process measures using explicit criteria. For instance, eye screening was considered to have occurred if the record contained “any mention of eye examinations by an eye care specialist (e.g., ophthalmologists, optometrists), including mention of an eye exam, diabetic eye exam, dilated eye exam, retinal exam, fundus or funduscopic eye exam, or examination or treatment of retinopathy…eye exams performed by primary care providers are NOT considered acceptable eye exams unless there is clear documentation that the patient’s eyes were dilated before the exam, and specific findings, or lack of thereof (e.g., retinal hemorrhage, proliferative retinopathy, macular edema, etc.) are clearly documented.”

Some complications were grouped for descriptive purposes after collection. Aggregate microvascular disease was defined as history of retinopathy, nephropathy, neuropathy, foot ulcer, or amputation. CVD was defined as history of coronary artery disease, stroke, transient ischemic attack, or peripheral vascular disease. Medication lists were abstracted and patients were categorized regarding use of specific classes of therapy (e.g., any lipid-lowering medication). Eye and foot screening were considered to have occurred if any documentation of their provision was found in the record.

Outcome measures included the five diabetes care elements most strongly supported by clinical trials evidence of benefit: change in rates of annual HbA_1c, LDL cholesterol, blood pressure, and eye and foot screening and change in the absolute values of HbA_1c, LDL cholesterol, and blood pressure. Change was defined as an increase or decrease in levels or proportions comparing the intervention year to the preintervention year. For HbA_1c, LDL cholesterol, and blood pressure, the most recent value obtained during the intervention year was compared with the most recent value obtained during the preintervention year. Provider use of the DMA was defined as any use within 1 week of a scheduled patient visit.

We examined baseline differences in patient and provider characteristics between groups using Student’s t tests and χ² tests. We analyzed outcomes of patients in the intervention group with those in the control group by intention-to-treat. Because the intervention was delivered at the physician level but outcomes were measured at the patient level, we accounted for clustering effects in all analyses. For change in mean levels or numbers of tests per year, we calculated the average level of these outcomes for each provider, averaged this provider average in each group weighted by the number of patients contributed per provider, further adjusted these averages for baseline levels to control for regression to the mean, and evaluated statistical significance using ANOVA models. For change in proportions over time, we used generalized estimating equations assuming compound symmetry for the between- and within-cluster correlation matrix (29). Significance of the preintervention to intervention year change in proportions was tested with a first-order time-by-group interaction term. The cluster size was 24 staff and resident providers in the intervention group and 42 providers in the control group. SAS statistical software (SAS Institute, Cary, NC) was used for all analyses (30) and P < 0.05 was considered statistically significant.

We estimated the power of the study for change in HbA_1c levels based on an intraclass correlation of 0.07 to inflate sample size estimates for clustering effects, assuming 80% power to detect differences between groups with a 0.05 two-sided significance level. We further assumed that there would be uniform, consistent use of the DMA in the intervention group. Detection of a 1.0% absolute difference in change in HbA_1c between groups required 127 patients per group; detection of a 0.5% difference required 486 patients per group.

Baseline staff provider and patient characteristics were similar comparing the intervention group with the control group (Table 1). Providers were a mix of men and women with a wide range of age and panel size. Approximately 7% of provider panels comprised patients with diabetes. Overall, providers believed they were familiar with diabetes management guidelines and usually applied them. Baseline patient characteristics were also well balanced between groups. The average study patient was aged in the late sixties, white, and insured by Medicare, with a duration of diabetes of ∼10 years. Compared with intervention patients, a greater proportion of control patients were treated with insulin or cholesterol-lowering medications. Hypertension was more prevalent in the patients in the intervention group, and 50% or more patients in both groups had at least one microvascular or CVD complication.

Preintervention levels of outcome measures and changes at the end of the intervention period are shown in Table 2. During the preintervention year, both groups had similar rates of glycemic and lipid control, but mean diastolic blood pressure was higher and rates of eye and foot screening were lower in the intervention group.

Rates of several process measures were improved with access to the DMA: the number of HbA_1c and LDL cholesterol tests obtained and proportions of patients with at least one foot examination per year increased modestly but significantly in the intervention group compared with the control group. The intervention had a modest but nonsignificant benefit on glycemic control; HbA_1c levels tended to improve in the intervention group and worsen in the control group. Lipid control improved in both groups, with somewhat larger but not statistically significant improvements in the intervention group compared with the control group. In the subgroup of patients taking lipid-lowering medication, proportions of intervention group patients with LDL cholesterol <130 mg/dl improved significantly compared with similar control group patients (30% vs. 10% increase, = 0.008). Blood pressure control generally improved slightly in both groups, although mean systolic blood pressure decreased in the control group but increased in the intervention group. Proportions of patients undergoing at least one eye examination per year increased only slightly more in the intervention group than in the control group.

Provider use of the DMA was variable. On average, providers viewed the DMA for 42% of scheduled patient visits. Four of 12 providers (33%) viewed the DMA for >70% of patient visits, 33% of providers used the DMA for 28–30% of patient visits, and 33% of providers used the DMA for 0–17% of patient visits. Nurses and nurse practitioners used the DMA for 0–13% of patient visits. In “treatment received” analyses (comparing control providers with intervention providers who used the DMA for more than the mean number of scheduled patient visits), overall intervention effects were similar to those found in intention-to-treat analyses (data not shown).

Common sense suggests that computer systems providing interactive patient-specific management support should improve care for complex, data-intensive diseases such as type 2 diabetes, but only limited evidence supports this notion. In this controlled trial, we demonstrated that availability of the DMA, a web-based information management and decision support interface, led to modest but significant improvements in several evidence-based processes of diabetes care, including increased rates of testing for levels of HbA_1c and LDL cholesterol and screening for foot disease. Access to the DMA also was associated with nonsignificant improvements in glycemic and lipid control. Although several studies have demonstrated benefits of computer systems used for prompting or writing orders (17), our study is among the first to suggest possible benefits of an interactive, patient-specific, web-based computer aid to chronic disease management.

Prior research suggests beneficial effects of diabetes disease management programs or computer decision support systems (17–19,22,24,31,32), but lack of rigorous control in many of these studies prevents reliable discrimination of intervention effects from the background secular improvements in levels of HbA_1c, blood pressure, and lipids occurring nationwide (33–36). The specific benefits of computer aids to diabetes care are gradually becoming rigorously documented. In one controlled trial, Peters and Davidson (23) showed that a comprehensive computerized tracking system led to significant improvements in glycemic and cholesterol control and in several process measures of quality diabetes care in a managed care setting. In a controlled study of provider-oriented, patient-specific diabetes decision support integrated into an EMR, Lobach and Hammond (37) demonstrated improvements in several process outcomes. We demonstrated modest improvements in process outcomes with computerized decision support but also evaluated metabolic outcomes. Process measures are an important element of diabetes care; for instance, increased rates of foot examinations associated with use of the DMA can be expected to lead directly to lower rates of limb-threatening foot lesions in AMC patients with type 2 diabetes (7). However, for metabolic process outcomes, it is improved risk factor levels that lead to reduced risk for complications. For glycemic control, we found suggestive trends in the intervention group in improvement in both HbA_1c levels, compared with worsening of glycemic control in the control group. From a population perspective, even the small 0.3% absolute difference in HbA_1c levels that developed between the groups is likely to confer benefit over time (38). Another interpretation of these findings, however, is that improved process outcomes (for instance, increasing HbA_1c testing rates) do not necessarily equate with improved metabolic control (for instance, lower HbA_1c levels). This distinction merits emphasis, because most current quality standards focus only on measuring and rewarding improvements in process (12).

Our study has several limitations. Low rates of some diabetes care activities in study practices may have been a function of incomplete (and then improved) documentation rather than inadequate (and then improved) delivery of care (39). However, for purposes of accountability, improved documentation is equivalent to improved care delivery. We also lacked substantive patient surveillance and self-care interventions; diabetes is a public health problem in which self-care is a major locus of disease control, but many patients remain only loosely affiliated with their primary care setting (40). Whether embedding the DMA within a larger program designed to enhance population surveillance and patient self-care might lead to even better patient outcomes is a focus of ongoing research (41). Finally, use of the DMA was variable and inconsistent among providers, contributing to its relatively modest effects. Based on poststudy feedback from intervention group providers, barriers to use included lack of integration into workflow (the DMA was not an obligatory prompting or reminder system), time pressure during the clinical encounter, the fact that diabetes was only one of many chronic problems to manage in a typical primary care encounter, and the belief among some providers that use of computers during an encounter is a barrier to effective patient-doctor communication. The fact that the DMA had any impact at all suggests that if these barriers can be addressed, web-based patient-specific decision support has the potential to produce more substantial improvements in patient outcomes.

The methods and findings of our intervention are potentially generalizable. The study was conducted in a pluralistic, mixed-payor group practice different than the highly integrated managed care settings in which most care improvement research has usually been conducted (18,19,22,23,31). The informatics architecture underlying the DMA is adaptable to local medical data sources and guideline implementations. The DMA is viewed with a web browser, and thus on virtually any computer, and while it can be integrated into existing medical record systems, it can also be used as a stand-alone clinical tool much like other web-based patient care resources.

Improving type 2 diabetes care is an important challenge. Excellent evidence of treatment efficacy has not been translated into effective care for many patients. From a provider perspective, the number and complexity of treatment goals, therapeutic interventions, and testing and screening requirements represent important barriers to delivery of comprehensive, high-quality care. In this controlled study, we demonstrated that web-based, patient-specific, interactive decision support has the potential to improve elements of care known to lead directly to reduced morbidity and mortality in type 2 diabetes.

This study was supported by a grant from the National Pharmaceutical Council and by the MGH Primary Care Operations Improvement and Clinical Research Programs. Funds for dissemination of results were also provided by Aventis Pharmaceuticals. J.B.M. received support from an American Diabetes Association clinical research grant and a junior faculty development grant from SmithKline Beecham.

We thank Barbara White for assistance with DMA training and provider usage supervision, Jo Ann David-Kasdan, RN, MS, for assistance with data abstraction and management, and Yuchiao Chang, PhD, for assistance with design of the statistical analysis.