Simulated Physician Learning Program Improves Glucose Control in Adults With Diabetes

This group-randomized trial was designed to test whether a personalized simulated learning intervention for physicians would improve care delivered to adults with uncontrolled diabetes.

The study was conducted from October 2006 to May 2007 at HealthPartners Medical Group (HPMG), a large medical group in Minnesota that serves about 230,000 patients. Eleven HPMG clinics were randomly selected and block randomized on the basis of baseline quality of diabetes care and number of consenting primary care physicians (PCPs) to either receive or not receive the intervention.

PCPs were eligible for the study if they practiced in one of the study clinics, provided care to at least 10 adult patients with diabetes, and signed a consent form. Patients were classified as having diabetes if they had two or more outpatient diabetes ICD-9 codes or used a diabetes-specific medication in the year before randomization (see Fig. 1 for the detailed consort description).

A group of PCP experts actively involved in diabetes guideline development identified and reached consensus on 25 essential clinical practices for the successful management of type 2 diabetes in adults (Table 1). The goal of the learning program was to promote mastery of these essential clinical practices.

The research team then used existing electronic medical record data to profile individual PCP performance on the essential clinical practices. If a physician performed below average compared with his or her physician peers on a clinical practice, a simulated case initialized with clinical parameters designed to teach the specific clinical practice was assigned.

Initialization parameters for the set of learning cases were defined to represent the wide range of complex clinical scenarios a PCP could encounter. Each case encompassed a different mix of baseline patient demographics (age, sex, duration of diabetes), medical history and comorbidities (congestive heart failure, renal insufficiency, coronary artery disease), pharmacologic use (active medications for glycemia, blood pressure [BP], lipids, depression, aspirin), clinical states (A1C, BP, lipid, creatinine, and self-monitoring of blood glucose results), and other subjective patient characteristics (hypoglycemia symptoms, medication adherence, lifestyle habits, and depressive symptoms). An automated case generator was developed to create distinct simulated learning cases.

A detailed description of the simulation software has been previously published (13). The patient model embedded in the software uses prespecified formulas derived from dose-response curves for drugs, lifestyle advice, and referrals. This allowed calculated changes in the patient state at each encounter on the basis of physician treatment actions. The interface mimics electronic medical record screens that permit the learner to prescribe drugs, order labs or diagnostic tests, make referrals, give patient advice, change frequency of recommended self-monitoring of blood glucose testing, view self-monitoring of blood glucose results, start or adjust insulin with each meal and at bedtime, and see the patient at any desired frequency for phone or office visits.

A key strategy of the learning intervention was the individualization embedded in the intervention at multiple levels. First, the PCPs received a customized set of simulated cases selected to address their assessed learning needs. Second, feedback resulted from a multitude of individual provider actions, yielding a unique trajectory to each case in response to the learner's specific treatment decisions. Learning feedback occurred through seeing the clinical effects of treatment moves at subsequent encounters, seeing graphic displays of the projected results of accumulated treatment actions, and direct textual feedback after each encounter consisting of a critique of past actions and suggestions for future ones.

PCPs were sent an email link to the program, accessed through a secure login page housed on an internal server. Clicking on the name of one of the 12 assigned cases started the simulation. The physician was challenged to achieve the American Diabetes Association–recommended treatment goals for BP, lipids, and A1C within 6 months of simulated time. Each case took about 15 min to complete.

Physicians received financial compensation ($600) and up to 6 h of CME credit for completing the program, which had to be done on their own time. The average number of days to complete the 12 cases was 5.5. At the end of the intervention, physicians also were sent an evaluation of the learning program.

Measures of intermediate outcomes (A1C, BP, LDL levels) of diabetes care in patients of providers extracted from the electronic medical record were the principal dependent variables. The baseline measurements were the clinical values viewable by the provider at the first encounter after completing the learning intervention. For A1C and LDL, this was the last clinical value obtained in the 1-year preintervention period. Because of point-of-care availability of blood pressure levels at encounters, the baseline measurement for BP was the first value in the 1-year postintervention period. For all measures, the last value in the postintervention period is selected as the postintervention variable. No changes in laboratory assay methods occurred during the study.

Costs were estimated from the health plan perspective and included the costs of the intervention and health care costs. Intervention costs included 1) marketing to physicians; 2) profiling physician learning needs; 3) implementing physician training, case assignment, and email reminders; and 4) physician compensation for completing cases. Health care costs included outpatient services and pharmaceuticals. Inpatient services were not included because the intervention was not expected to affect hospitalization rates. Costs were estimated using methods previously developed to assign costs of services in this medical setting (14). Services were based on relative value units assigned on the basis of procedure codes recorded and priced at the 2009 physician services conversion factor of $36.07. Costs of pharmaceuticals were based on 68% of the average 2009 wholesale price.

The independent variable was an indicator for study arm. The interaction of this with time was used to assess the differential impact of the intervention on the prespecified outcomes. Because the trial was group-randomized at the clinic level, imbalance in patient characteristics was expected. Patient-level independent variables include age, sex, race, and validated indicator variables for coronary artery disease and congestive heart failure. Adults older than 80 years and individuals with Charlson comorbidity scores of ≥3 (indicating high short-term risk of mortality) were excluded from the study because of legitimate debate about appropriate clinical goals in such scenarios (15).

Attributes of study-consented physicians and patients linked to these physicians are compared by study arm using descriptive statistics (mean, SD, proportions) and using independent sample t tests, contingency tables, and Pearson χ² tests.

General and generalized linear mixed models with a repeated time measurement (baseline and postintervention) were used to analyze continuous and binary outcomes using SAS Proc Mixed and Proc Glimmix. These models included a term for study arm, time (baseline or postintervention), a time by study arm interaction term, and random intercepts to account for multiple levels of nesting. The time-by-study-arm interaction term tested the effect of the intervention arm over time relative to the effect of the control arm over time. The analyses on test values were also conducted predicting postintervention values from study arm, preintervention test value, and patient covariates.

Denominators for the analysis of test rates, encounter rates, and numbers of tests and encounters include all (n = 3,417) eligible patients linked to study-consenting physicians. Analyses for change in values (e.g., A1C) are based on subsets of patients who are not at goal at baseline on particular measures of interest, because they are targeted in the intervention. A priori sample size calculations assumed an analytic sample of 500 diabetic patients per study arm, based on 20 providers with 25 diabetic patients not at A1C goal. Effective patient sample size was estimated as n = 291 per arm due to clustering of patients within physicians (estimated intraclass correlation [ICC] = 0.03). This study was designed with 80% power to detect an A1C difference of 0.3% between study arms, with a two-tailed α = 0.05. Alpha levels are not adjusted for testing of multiple dependent variables.

Generalized linear models, assuming γ distribution and a log link function, were used to analyze the relationship between costs and study arm (16). These models included a term for study arm, time (baseline or postintervention), and a time-by-study-arm interaction term. A standardized estimate of the effect of the intervention on costs was calculated as a mean difference in predicted costs among study patients (including all patients with A1C >7% in the preintervention period), as they were alternatively assigned to the intervention and control groups in the post period. SEs were estimated using the nonparametric bootstrap, and significance values were computed using the percentile method (17).

The study was reviewed in advance, approved, and monitored on an ongoing basis by the HealthPartners Institutional Review Board, Project #03-083.

Attributes of study-eligible patients and PCPs are presented in Table 2. Diabetic patients' age was 56.4 ± 10.7 years (mean ± SD), 24.0% were 65–74 years old, 48.9% were female, and 29.2% were nonwhite. At baseline (first preintervention value), 47.5% had A1C <7%, 61.7% had systolic BP (SBP) <130 mmHg, 67.3% had diastolic BP (DBP) <80 mmHg, and 60.2% had LDL <100 mg/dl. The number of diabetic patients per study-enrolled PCP ranged from 10 to 125, with a mean of 56.4 ± 29.1. Randomization at the clinic level resulted in an intervention arm with a higher proportion of younger and male patients.

In four-level random intercept models (measurement occasion nested within patient, provider, and clinic), ICCs at the clinic level were generally small, with values of ICC <0.0001 for A1C and LDL value, ICC = 0.001 for SBP value, and ICC = 0.003 for DBP. Because of the low level of variance at the clinic level, three-level models are presented by dropping the random intercept term for clinic.

Table 3 shows baseline and follow-up measures of diabetes encounters, A1C, BP, and LDL test rates. They increased between pre- and postintervention measurements, with no differential effect for the intervention group, as shown by the nonsignificant time-by-condition interaction term P values.

Statistically significant improvements over time were seen within study arm for A1C, SBP, DBP, and LDL values. Intervention-arm patients had a significantly greater improvement (decrease) in A1C value from baseline to postintervention than the control arm by a value of −0.19% (95% CI −0.37 to −0.01, P = 0.034). Among patients with an A1C ≥7% at the last preintervention measurement, 29.2% of intervention-arm and 22.5% of control-arm patients had an A1C <7% at the last postintervention measurement (P = 0.0099, n = 1,403). There were no significant differences in SBP or DBP reduction over time by study arm. Control-arm patients had a significantly greater improvement (decrease) in LDL value from baseline to postintervention than the intervention arm by a value of −5.1 mg/dl (95% CI −9.8 to −0.3, P = 0.039). Analysis of the proportion of patients at LDL goal (<100 or <70 mg/dl with coronary artery disease) at the last measurement in the 12-month postintervention period showed no significant difference (0.413 intervention clinics, 0.414 control clinics, P = 0.99, n = 1,069).

Intervention costs were estimated to be $27 per patient. Total costs, including intervention and health care costs, were estimated to be $71 (SE = 142, P = 0.63) lower per patient in the intervention clinics compared with control clinics.

A total of 85% of intervention PCPs completed a survey of satisfaction and self-assessed impact of the learning. Of these, 88% would recommend the learning experience to other physicians, 82% thought it would help most doctors improve their diabetes care skills, 82% reported they would be more likely to intensify medication for their diabetic patients, 59% would shorten their visit intervals, and 18% would start insulin more often. A separate analysis showed that preintervention PCP performance measured by the percentage of patients at glycemic goal was not a significant predictor of A1C postintervention (P = 0.98).

This personalized physician learning intervention demonstrated a modest but significant A1C lowering without increasing patient visits or total net costs. The observed A1C impact (a 0.5% improvement in A1C in the intervention group, which was 0.19% better than the control group) is of roughly the same magnitude as that reported in uncontrolled observational studies of more expensive interventions intended to improve diabetes care, such as clinical information systems (18), patient education, and disease management programs (19). The net A1C improvement is clinically significant based on the potential to reduce patient complications, as demonstrated through UK Prospective Diabetes Study results showing 37% lower microvascular complications for every 1% absolute A1C reduction (20).

PCPs reported high levels of satisfaction with the intervention, repeated learning cases voluntarily, and reported tangible changes in the way they manage diabetes. The learning effect of the computer-based intervention was present independent of PCP baseline diabetes performance and indicates an ability to transfer what was learned from the simulated cases to the care of real patients—an important challenge and desirable finding in simulation research.

There are a number of points to make about the cost of this intervention in relation to its modest clinical effectiveness. First, the intervention did not increase patient visits or testing rates in real patients and trended toward cost-saving. Second, this simulated approach to physician learning is compatible with a wide range of chronic care improvement activities designed to activate patients or develop prepared care teams (21). Finally, the simulation content can be easily adapted to accommodate changes in care guidelines, discourage unnecessary tests or treatments, and encourage use of generics and might help modulate to some degree the high costs of diabetes care. The marginal costs for ongoing use of this learning tool are relatively small and principally involve periodic updates to ensure that the simulation model remains current with evidence-based treatment strategies and newly approved drugs.

Future research should investigate issues that might increase the impact, reduce the complexity, or broaden the dissemination potential of the intervention. Our use of electronic medical record–based physician treatment patterns to assess learning needs is innovative but can be difficult. Development of simpler methods to evaluate physician learning needs (e.g., assessment on a set of prespecified simulated cases) warrants investigation. Second, the intervention needs adaptation for web distribution and to simplify addition of clinical updates when needed. Third, at the time of this study, generalized diabetes care goals, such as A1C of <7%, were widely accepted. As personalized medicine seeks to accommodate individualized treatment goals, this approach provides an opportunity to teach a systematic approach to setting clinical care goals individualized to patient characteristics such as comorbidities, polypharmacy concerns, hypoglycemia risk, and genetics (22,23).

Additional work is needed to elucidate more precisely the mechanisms responsible for the observed effects. The variable effect on clinical domains may be partially related to primary emphasis on glucose-related feedback in the learning cases relative to BP or lipid-related feedback. Recent clinical trial results (23,24) highlight the importance of directing more attention to BP- and lipid-management issues. It is notable that the study site had relatively good baseline levels of A1C, BP, and LDL (25). The impact of this learning intervention in settings with worse baseline levels of diabetes care remains to be determined. This promising care-improvement strategy might also be applied to other clinical domains.

Despite some limitations, these data demonstrate that delivery of a brief and relatively inexpensive individualized physician learning intervention improved intermediate outcomes of diabetes care without increasing costs. Experimentation with simulated personalized learning interventions in a broad range of other care or educational settings seems warranted.

This project was funded by NIDDK Grant #R01 DK068314 to HealthPartners Research Foundation.

J.M.S.-H. received support indirectly through HealthPartners Research Foundation for multisite drug trials funded by Merck, GlaxoSmithKline, and Abbott Pharmaceuticals and from Merck for a randomized trial on educational methods for patients with diabetes. H.L.E. owns stock in Pfizer. No other potential conflicts of interest relevant to this article were reported.

J.M.S.-H., P.J.O., P.E.J., T.G., and S.E.A. contributed to writing the manuscript. J.M.S.-H., P.J.O., W.A.R., P.E.J., T.G., S.E.A., and H.L.E. contributed to the review and editing of this manuscript.

An abstract of the main results of this study was presented at the 2010 American Diabetes Association Scientific Sessions.