OBJECTIVE

To improve outcomes of patients with adult type 2 diabetes by decreasing HbA1c undertesting, reducing the proportion of patients with poor glycemic control, and lowering mean HbA1c levels using a quality improvement (QI) program.

RESEARCH DESIGN AND METHODS

Six years of outpatient electronic health record (EHR) data were analyzed for care gaps before and 2 years after implementing a QI initiative in an urban academic medical center. QI strategies included 1) individual provider and departmental outcome reports, 2) patient outreach programs to address timely follow-up care, 3) a patient awareness campaign to improve understanding of achieving clinical goals, 4) improving EHR data capture to improve population monitoring, and 5) professional education.

RESULTS

Analysis (January 2010 to May 2018) of 7,798 patients from Tulane Medical Center (mean age 61 years, 57% female, 62% black, 97% insured) with 136,004 visits showed target improvements. A Cox proportional hazards model controlling for age, sex, race, and HbA1c level showed a statistically significant reduction in HbA1c undertesting >6 months (hazard ratio 1.20 ± 0.07). Statistical process control charts showed 15.5% relative improvement in the patient proportion with HbA1c >9% (75 mmol/mol) from 13% to 11% (P < 10−6) following QI interventions and a 2.1% improvement of population mean HbA1c from 7.4% (57 mmol/mol) to 7.2% (55 mmol/mol) (P < 10−6).

CONCLUSIONS

Multidisciplinary QI teams using EHR data to design interventions for providers and patients produced statistically significant improvements in both care process and clinical outcome goals.

Around the world, significant numbers of patients with type 2 diabetes fail to meet evidence-based therapeutic guidelines despite increasing disease knowledge, improving treatment options, new technologies, more patient education programs, and better understanding of the psychosocial determinants of health (15). To address this situation, many models for improvement have been proposed or deployed based on inspection and regulation, financial incentives, market competition and consumer choice, and process improvement and system change. The first three approaches have widespread support and implementation in health care, but, unlike in the industrial and service sectors, health care has underused process improvement and system design strategies to improve care quality despite compelling advocacy by physicians (68) and health organizations (9,10) for more than 50 years.

To improve care quality for all patients with diabetes, the American Diabetes Association (ADA) has developed the Diabetes INSIDE (Inspiring System Improvement with Data-Driven Excellence) program based on the Standards of Medical Care in Diabetes (11) and established quality improvement (QI) principles and practices (12). Diabetes INSIDE is a multiyear, system-wide, structured program of population health analytics, care system redesign, and provider and patient training, education, and support implemented at leading health care systems across the country (13).

As a participant in Diabetes INSIDE, Tulane Medical Group reviewed historical electronic health record (EHR) data on clinical process and outcome data before establishing three objectives: 1) to reduce the number of HbA1c tests delayed >6 months, 2) to reduce the proportion of patients with HbA1c >9% (75 mmol/mol), and 3) to reduce the overall population mean HbA1c by deploying multiple educational and workflow QI interventions targeting both staff and patients across primary care departments with the goal of establishing measurable and significant change in these objectives.

Study Population and Setting

The enrolled population (Table 1) were all adults (≥18 years old) diagnosed with type 2 diabetes from May 2010 to June 2018 in the outpatient Family Medicine, Internal Medicine, and Endocrinology practices at Tulane Medical Center, an urban academic medical center serving the metropolitan New Orleans area. The population attending the practices has a high proportion of African Americans and comes from diverse socioeconomic groups. There were no exclusions on patients entering or exiting Tulane care at any time or on care duration at Tulane.

Table 1

Demographic and clinical characteristics of patients seen before and after QI interventions

All visitsBaseline visitsFollow-up visits
1/2010 to 5/2018Before 6/2016After 6/2016
(n = 136,004)(n = 97,035)(n = 38,969)
Patients, n 7,798 6,851 4,663 
Visits per patient    
 Mean, n (SD) 14 (12) 14 (12) 13 (12) 
 Median, n 11 11 10 
Age, years    
 Mean (SD) 61 (13) 60 (14) 61 (13) 
 Median 62 62 63 
Sex, n (%)    
 Female 4,354 (57) 3,748 (57) 2,542 (56) 
 Male 3,357 (44) 2,825 (43) 2,021 (44) 
Ethnicity/race, n (%)    
 Non-Hispanic    
  Black 4,683 (62) 4,193 (63) 2,846 (64) 
  White 2,541 (34) 2,201 (33) 1,433 (32) 
 Other 297 (4) 157 (4) 164 (4) 
Insurance, n (%)    
 Commercial 4,405 (57) 3,762 (56) 2,641 (58) 
 Medicare 2,851 (37) 2,607 (39) 1,714 (38) 
 Uninsured 212 (3) 173 (3) 108 (2) 
 Veterans 158 (2) 131 (2) 72 (2) 
 Medicaid 66 (1) 61 (1) 17 (0) 
Smoker, n (%)    
 Never 2,295 (56) 1,798 (60) 1,416 (54) 
 Former 1,049 (25) 628 (21) 721 (28) 
 Current 779 (19) 562 (19) 484 (22) 
Visit BMI (kg/m2), n (%)    
 25 17,719 (12) 12,330 (12) 5,389 (13) 
 25–25.9 33,518 (23) 23,412 (23) 10,106 (25) 
 >30 80,706 (56) 57,374 (55) 23,332 (57) 
BP, mean (SD), mmHg    
 Systolic 134 (21) 134 (21) 135 (21) 
 Diastolic 75 (12) 75 (12) 74 (12) 
Hypertension diagnosis, n (%)    
 No 83,655 (58) 60,049 (58) 23,606 (58) 
 Yes 51,063 (35) 36,560 (35) 14,503 (35) 
Cardiovascular disease diagnosis, n (%)    
 No 140,856 (86) 100,418 (85) 40,438 (88) 
 Yes 22,819 (14) 17,153 (15) 5,666 (12) 
HbA1c, %    
 Mean (SD) 7.5 (1.7) 7.5 (1.8) 7.4 (1.7) 
 Median 7.0 7.1 7.0 
HbA1c interval, days    
 Mean (SD) 251 (243) 271 (264) 179 (121) 
 Median 179 183 151 
LDL cholesterol, mg/dL    
 Mean (SD) 100 (38) 100 (38) 98 (39) 
 Median 94 94 91 
LDL cholesterol interval, days    
 Mean (SD) 292 (237) 306 (249) 215 (129) 
 Median 226 239 188 
All visitsBaseline visitsFollow-up visits
1/2010 to 5/2018Before 6/2016After 6/2016
(n = 136,004)(n = 97,035)(n = 38,969)
Patients, n 7,798 6,851 4,663 
Visits per patient    
 Mean, n (SD) 14 (12) 14 (12) 13 (12) 
 Median, n 11 11 10 
Age, years    
 Mean (SD) 61 (13) 60 (14) 61 (13) 
 Median 62 62 63 
Sex, n (%)    
 Female 4,354 (57) 3,748 (57) 2,542 (56) 
 Male 3,357 (44) 2,825 (43) 2,021 (44) 
Ethnicity/race, n (%)    
 Non-Hispanic    
  Black 4,683 (62) 4,193 (63) 2,846 (64) 
  White 2,541 (34) 2,201 (33) 1,433 (32) 
 Other 297 (4) 157 (4) 164 (4) 
Insurance, n (%)    
 Commercial 4,405 (57) 3,762 (56) 2,641 (58) 
 Medicare 2,851 (37) 2,607 (39) 1,714 (38) 
 Uninsured 212 (3) 173 (3) 108 (2) 
 Veterans 158 (2) 131 (2) 72 (2) 
 Medicaid 66 (1) 61 (1) 17 (0) 
Smoker, n (%)    
 Never 2,295 (56) 1,798 (60) 1,416 (54) 
 Former 1,049 (25) 628 (21) 721 (28) 
 Current 779 (19) 562 (19) 484 (22) 
Visit BMI (kg/m2), n (%)    
 25 17,719 (12) 12,330 (12) 5,389 (13) 
 25–25.9 33,518 (23) 23,412 (23) 10,106 (25) 
 >30 80,706 (56) 57,374 (55) 23,332 (57) 
BP, mean (SD), mmHg    
 Systolic 134 (21) 134 (21) 135 (21) 
 Diastolic 75 (12) 75 (12) 74 (12) 
Hypertension diagnosis, n (%)    
 No 83,655 (58) 60,049 (58) 23,606 (58) 
 Yes 51,063 (35) 36,560 (35) 14,503 (35) 
Cardiovascular disease diagnosis, n (%)    
 No 140,856 (86) 100,418 (85) 40,438 (88) 
 Yes 22,819 (14) 17,153 (15) 5,666 (12) 
HbA1c, %    
 Mean (SD) 7.5 (1.7) 7.5 (1.8) 7.4 (1.7) 
 Median 7.0 7.1 7.0 
HbA1c interval, days    
 Mean (SD) 251 (243) 271 (264) 179 (121) 
 Median 179 183 151 
LDL cholesterol, mg/dL    
 Mean (SD) 100 (38) 100 (38) 98 (39) 
 Median 94 94 91 
LDL cholesterol interval, days    
 Mean (SD) 292 (237) 306 (249) 215 (129) 
 Median 226 239 188 

Demographic data, except age, are per patient, whereas other values are per visit averaging multiple values per patient.

The Tulane University Medical Group has a history of executing other clinical QI projects with the support and cooperation of faculty, staff, and management. For this study, Tulane established a 19-member multidisciplinary team of clinicians, nurses, pharmacists, dietitians, data analysts, and managers from the three participating departments to define, measure, analyze, and implement interventions to improve diabetes care quality. The ADA provided additional data analysis, QI training, education, coaching, and consultation throughout the 24 months of the project.

Interventions

Visit-level outpatient eClinicalWorks EHR records were queried from May 2010 to June 2016 at study start to establish a historical baseline of medical system performance and requeried from May 2010 to June 2018 at study conclusion for follow-up outcomes analysis against the ADA guidelines.

From the baseline analysis, the QI team selected goals to improve the frequency of HbA1c testing to meet guideline recommendations as the primary improvement objective and to reduce population HbA1c levels as the secondary objective. The team planned multifaceted interventions targeting health system changes, health care providers, and patients in the Internal Medicine, Family Medicine, and Endocrinology departments (14).

Health system interventions began in October 2016 and included 1) a provider outreach program of departmental and individualized actionable data reports on HbA1c testing intervals and outcomes, 2) a patient outreach program to address individual timely follow-up care, 3) a patient awareness campaign to improve understanding of diabetes and the importance of following and improving clinical outcomes, 4) provider training and resources for improving population data capture in the EHR, and 5) professional education on the 2016 and 2017 ADA guidelines. As QI interventions do not require patient consent and received institutional review board waiver, patients were not explicitly informed of QI interventions, although some might become aware of new or additional patient outreach and education initiatives.

The provider outreach program consisted of unblinded monthly performance reports by provider and department. Reports color-coded (red: out-of-compliance; yellow: borderline compliance; green: in-compliance) clinical goals and specified individual patient outcomes for HbA1c and days since the last HbA1c test as well as individual provider outcomes for the percentage of patients with HbA1c <8% (64 mmol/mol) and blood pressure (BP) <140/90 mmHg, median days between HbA1c tests, and number of HbA1c tests <180 days, 180–364 days, and >365 days for patients below and above HbA1c of 8% (64 mmol/mol). Monthly departmental reports presented population run charts over the last year for the same measures as well as HbA1c >9% (75 mmol/mol).

The patient outreach program consisted of the Population Health Team Patient Care Coordinators and Clinic Nurse Navigators conducting an analysis of their diabetes population and instituting a telephone and mail outreach reminder system for laboratory and follow-up visits.

The patient awareness campaign developed a Diabetes Personal Health Record in the EHR summarizing the history of a patient’s vital signs, appointments, foot and eye examinations, immunizations, laboratory results, and referrals for diabetes management that would be distributed and reviewed with each patient scheduled for a clinic visit or diabetes class. In addition, each patient would receive A.D.A.M.’s comprehensive diabetes patient education materials that are built into the eClinicalWorks EHR (15).

For provider training and resources to improve population health quality tracking, the QI team conducted department group and one-on-one training sessions and prepared how-to documents to train staff on the importance and value of standardizing EHR data capture for manually entered diabetes care data such as point-of-care HbA1c tests and foot examinations.

Provider professional education included three grand rounds over 24 months by members of the ADA’s senior staff and professional members from the ADA Professional Practice Committee on the diabetes guidelines and on local clinical outcomes compared with the guidelines.

Measures

EHR data that included patient demographics, insurance, provider, referrals, vital signs, laboratory data, medications, health maintenance, and diagnostic and procedure codes at each visit were analyzed to measure the effects of QI interventions on population health outcomes.

HbA1c values for each month were aggregated to compute monthly mean and SD of the population HbA1c and the monthly proportion of patients with HbA1c >9% (75 mmol/mol). The number of days between consecutive HbA1c test dates for each patient was calculated, and the test intervals were used for recurrent time-to-event analysis. HbA1c percentage change between consecutive visits was calculated for each patient and averaged at monthly intervals over the population.

EHR data cleaning consisted of rejecting a small number of observations with incompatible data types as “missing,” for example, text commentary for laboratory values, or physiologically unachievable values from manual entry errors, especially for height, weight, BP, and point-of-care HbA1c. Any multiple daily values were replaced by the median value. Valid fields of incomplete patient observations were used as is; no imputation was done for missing data. Aggregate statistics on rational subgroups with <10 observations were also excluded from analysis. Only patients ≥18 years old were included in the analysis.

Qualitative assessment of the interventions was assessed by departmental advice and feedback during the implementation phase.

Data Analysis

The Tulane University Medical Group’s Population Health Data Analyst extracted EHR data into Excel worksheets, which were then analyzed in R software (16) with custom scripts.

HbA1c testing intervals were analyzed as recurrent events for each patient using a nonparametric Kaplan-Meier model and log-rank statistics as well as a right-censored Cox proportional hazards model (17,18) controlling for age, sex, race, HbA1c level, and time effects. To account for the time dependence of HbA1c and predictors before and after 6 months, a Heaviside step function (0 if ≤6 months; 1 if >6 months) was added as an interaction term (19). The semiparametric Cox model was stratified by clinical department as each department had nonproportional time-to-event HbA1c intervals. Other candidate predictors that did not satisfy the proportional hazards assumption assessed using Schoenfeld residuals against time (20) were rejected. Accounting for possible correlations among multiple HbA1c testing intervals per patient using robust variance estimates did not affect the results. Any intraprovider correlations were subsumed by the nonparametric underlying baseline hazard function of the Cox model.

HbA1c population outcomes were assessed with statistical process control (SPC) charts using 3-σ control limits and Western Electric rules for distinguishing special and common cause variation (21). SPC charts are designed to detect process outliers, shifts, and instability. Means before and after interventions were compared using t tests. SPC charts plot the monthly means about the population mean (horizontal line) and ±3-σ control limits (stepped horizontal lines). The control chart mean estimates the population mean from the sample mean at each time point, while the control limits are estimates of the SE of the population mean estimates. Therefore, the vertical spacing of the control limits varies proportionately with the sample SD and inversely with the square root of the number of samples in a month.

The Tulane QI team independently prepared run charts stratified by department for the monthly provider performance feedback reports, but they were not used for analyzing the outcomes of the interventions. Providers reviewed individual patient HbA1c testing intervals at the point-of-care as color-coded numeric values.

Ethical Considerations

The Tulane Institutional Review Board reviewed and approved the activities in this QI study, without requiring individual patient informed consent.

The multidisciplinary QI team review of >6 years (January 2010 to May 2016) of historical baseline data identified care quality gaps in informatics monitoring of ongoing care, poor HbA1c testing frequency and control, ineffective control with many diabetes drug regimens, and significantly higher mean HbA1c levels in younger patients. Care quality gaps were not seen with subgroup analysis by socioeconomic factors. The identified gaps were presented as the first of three didactic QI interventions at a combined Family Medicine, Internal Medicine, and Endocrinology grand rounds along with an outside speaker reviewing the ADA Standards of Medical Care in Diabetes to establish community support for introducing interventions to address the care quality gaps.

The QI team voted to reduce HbA1c testing frequency as the primary quality aim and to reduce HbA1c levels in both the overall population and patients with HbA1c >9% (75 mmol/mol) as the secondary QI aims. The team hypothesized that improving a process measure, HbA1c testing frequency, was a tangible, measurable goal that would improve patient outcomes and that would be quickly affected by the interventions planned across three separate clinic locations. The initial project timeline for rollout was estimated as 3 months, but local staffing and resource issues as well as needed local adaptations and modifications slowed complete rollout by 6 months for two of the clinics.

Table 1 reports the overall population characteristics, as well as before and after the interventions, over 8.3 years. Patient, sociodemographic, and diagnostic characteristics—patients, visits, age, sex ethnicity/race, insurance, smoking, BMI, hypertension, and cardiovascular disease—were stable. However, clinical process (testing frequency) and outcome measures for HbA1c and LDL cholesterol improved significantly after the interventions.

ADA guidelines recommend quarterly testing in patients with changing treatments or who are not meeting glycemic goals and semiannual testing for patients with stable treatments and at therapeutic goals (22). Initial analysis of testing intervals showed that 47% of all patients had HbA1c intervals >6 months (Table 1) and that 13% of patients had intervals >2 years (Supplementary Fig. 1). Kaplan-Meier graphs of the proportion of patients not yet receiving an HbA1c test against the time interval from the last test (Supplementary Fig. 1) in the baseline/follow-up phases showed no differences in HbA1c test frequency <180 days, but did show a significant reduction in tests >180 days after intervention (P < 0.0001).

To quantify HbA1c testing interval improvement and to account for other influencing factors, the baseline/follow-up intervention test interval data were fit with a Cox regression model. Figure 1 shows the model coefficients and hazard ratios (HRs) for the predictor variables. Note the QI goal is shorter testing intervals, which implies HRs >1. Follow-up HbA1c testing intervals >180 days decreased by 20% (HR 1.20 ± 0.07) compared with baseline testing intervals. Men were 6% more likely than women to have decreased testing intervals (HR 1.06 ± 0.03). Compared with blacks, whites were 8% more likely to have longer testing intervals (HR 0.92 ± 0.03). After 180 days, the testing delay increased by ∼5% for each percentage increase in the HbA1c level (HR 0.95 ± 0.01). The nonparametric Kaplan-Meier time-to-event data (not shown) showed a higher proportion of patients with testing delays in Family Medicine compared with Internal Medicine. Endocrinology-managed patients had testing delays intermediate between the primary care departments perhaps because Endocrinology comanaged challenging referrals from both departments. Because the QI interventions were uniform across departments and departmental or provider covariates did not satisfy Cox modeling assumptions as a quantitative predictor, the testing intervals were stratified across the three departments.

Figure 1

Forest plot of Cox regression analysis HRs (black squares) showing significant improvement in HbA1c testing intervals during follow-up (1.20) while controlling for effects of HbA1c, age, sex (F, female; M, male), and race. 6m, 6 months. Error bars (horizontal lines) are 95% CIs of HRs. The right column shows P values for predictors in the Cox model. *P < 0.05; ***P < 0.001.

Figure 1

Forest plot of Cox regression analysis HRs (black squares) showing significant improvement in HbA1c testing intervals during follow-up (1.20) while controlling for effects of HbA1c, age, sex (F, female; M, male), and race. 6m, 6 months. Error bars (horizontal lines) are 95% CIs of HRs. The right column shows P values for predictors in the Cox model. *P < 0.05; ***P < 0.001.

Close modal

Figure 2 shows the relationship between HbA1c testing intervals and the percentage change in glycemic control since the last test (23). Patients tested less frequently than twice a year have increasingly worse glycemic control with lengthening test intervals compared with patients who are stable or improve with ADA guideline-recommended testing intervals. After 12–15 months, worsening glycemic control with increasing test intervals appears to plateau.

Figure 2

Mean percentage change in population HbA1c levels as a function of the time interval between HbA1c tests (n = 20,486 intervals). The right-most testing interval marked “25+” averages all intervals >24 months. The smooth line is a generalized additive model showing the trend of HbA1c changes with testing interval. The gray region is the 95% CIs of the model.

Figure 2

Mean percentage change in population HbA1c levels as a function of the time interval between HbA1c tests (n = 20,486 intervals). The right-most testing interval marked “25+” averages all intervals >24 months. The smooth line is a generalized additive model showing the trend of HbA1c changes with testing interval. The gray region is the 95% CIs of the model.

Close modal

The suite of QI interventions also significantly decreased the proportion of patients with poor HbA1c control from 13% to 11% (P < 0.001) as well as the overall population mean HbA1c from 7.4% to 7.2% (57–55 mmol/mol) (P < 0.001) as shown by the control charts (Fig. 3). Similar to HbA1c testing, there were differences in interdepartmental baseline measures, but all departments had temporally stable measures in the baseline period, and two departments showed statistically significant improvement following interventions—Family Medicine 19% to 15% (P < 0.001) and Internal Medicine 10% to 8% (P < 0.001). Endocrinology also improved, but not statistically significantly, 17% to 15% (P < 0.07).

Figure 3

Shewhart control charts of monthly proportion of patients with HbA1c >9% (75 mmol/mol) (A) and monthly mean population HbA1c (B) over 99 months showing sustained statistical improvement over the last 2 years. The center line shows the mean value, and the stepped horizontal lines represent ±3-σ upper and lower control limits. Open triangles mark monthly values outside control limits. Open circles mark statistically significant runs of 7 or more months of values above or below the mean. The dashed vertical line represents the start of the QI program. See Supplementary Table 1 for monthly and yearly sample sizes.

Figure 3

Shewhart control charts of monthly proportion of patients with HbA1c >9% (75 mmol/mol) (A) and monthly mean population HbA1c (B) over 99 months showing sustained statistical improvement over the last 2 years. The center line shows the mean value, and the stepped horizontal lines represent ±3-σ upper and lower control limits. Open triangles mark monthly values outside control limits. Open circles mark statistically significant runs of 7 or more months of values above or below the mean. The dashed vertical line represents the start of the QI program. See Supplementary Table 1 for monthly and yearly sample sizes.

Close modal

The HbA1c control charts (Fig. 3) begin showing evidence of statistical improvement in population HbA1c during 2015 before the start of this QI study. Prior to starting this diabetes QI study, Internal and Family Medicine implemented a QI program to improve hypertension management. Both significant overlap of the targeted populations and similar clinical care improvements would be synergistic in both groups. However, an SD control chart of the mean HbA1c (Supplementary Fig. 2) does not show statistically significant improvement until mid-2017 after the implementation of the diabetes-specific interventions, indicating additional improvement from these interventions through decreasing the population variance.

These results demonstrate that a QI initiative such as Diabetes INSIDE not only improves the process of care for people with diabetes but is also associated with a statistically significant 20% population-level improvement in HbA1c undertesting, a 15.5% improvement in the proportion of patients with HbA1c >9% (75 mmol/mol), and a 2.1% improvement in population mean and SD of HbA1c. Other QI studies have found comparable improvements in process and clinical diabetes measures (2426).

Fundamental to diabetes management is monitoring and regulating glycemic control with timely HbA1c testing (22). Previous studies have documented both over- and undertesting as well as inappropriate testing across differing populations and regions (27). However, many studies have documented undertesting in 17–60% of patients (27,28).

The primary aim of this study was improving HbA1c undertesting for the 47% of diabetes patients who received tests less frequently than every 6 months as recommended by the ADA guidelines. The range of delays in testing was substantial, with ∼20% of patients delaying >1 year and some many years. The QI team designed its interventions to target both the care team and patients with education, reminders, feedback, and workflow changes emphasizing guideline testing recommendations, the value of timely HbA1c testing in diabetes management, and decreasing opportunities for delayed testing at clinical encounters. These interventions resulted in a statistically significant 20% reduction (Fig. 1) in patients with tests delayed >6 months.

Questions of the sustainability of QI initiatives are well founded. Twenty years ago, Tulane published a similar successful QI intervention to improve the frequency of HbA1c testing in Medicare patients (29) that did not endure over the decades. This previous study also involved the Internal Medicine, Endocrinology, and Family Medicine departments and used interventions similar to the current study. Like this study, Internal Medicine and Endocrinology had the largest improvements, with Family Medicine in last place. However, the median population HbA1c in the current study, 7.1–7.0% (54–53 mmol/mol) (Table 1), was significantly improved compared with 8.5–7.8% (69–62 mmol/mol) in the prior study. Further, while the previous study was conducted in Medicare patients, the current study included a more diverse patient population.

Key to sustaining QI is long-term measuring and monitoring of achieved gains to detect in real-time unwanted changes introduced by subsequent intended or unintended clinical changes in practice. Widespread use of clinical informatics over the last decade has reduced the measuring challenge to QI sustainability, but the scarcity of frequent SPC reports to and QI training for direct-care providers is still a limitation for the long-term sustainability and continuity of QI initiatives.

This study also finds a similar association between HbA1c testing frequency and poorer glycemic control (Fig. 2) as reported by Driskell et al. (23) in the U.K. and poorer HbA1c, lipid, and BP control as recently reported (4). Many factors can contribute to testing noncompliance, and addressing each is challenging, but this study shows that a combination of provider and patient feedback along with a previsit reminder outreach can make a quantifiable improvement in reducing testing delays and a clinically significant improvement in patient outcomes.

More than 6 years of historical data from this system also showed that the proportion of patients with HbA1c >9% (75 mmol/mol) (Fig. 3A) and the overall population mean HbA1c (Fig. 3B) were statistically unchanging for years until the multidisciplinary QI team introduced a series of workflow changes in this study and a prior hypertension management study that statistically improved these process and outcome metrics over the 2-year follow-up interval. Besides changing central tendency measures, QI programs can also aim to reduce outliers (e.g., comorbidities and variance in populations). This study also found a statistically significant reduction in the SD of mean HbA1c during the last year of the follow-up (Supplementary Fig. 2), representing an overall narrowing in the spread of the population HbA1c values about the mean as a result of the additional QI interventions in this study supplementing prior QI efforts.

Despite the continuing introduction of novel new therapies and technologies, evolving evidence-based guidelines, broadly disseminated consumer awareness campaigns, extensive patient and provider education programs, and improving health care coverage, large proportions of patients do not achieve clinical goals (4,30), and population quality metrics remain stagnant (31). Many underlying factors contribute to this situation, but this study supports the QI premise that changing processes and systems has more influence on population outcomes than initiatives focused on only individuals (10).

The relative cause-and-effect relationships of the many factors introduced and influencing changes in a QI study cannot be assessed separately with observational studies, and to many health care professionals, this is a reason to dismiss such studies as lacking rigor and generalizability. Additional misunderstanding comes from the different objectives in different types of observational studies. Many observational studies compare the before/after outcomes of known cohorts of patients and providers, while other studies, like this one, do not track specific patients and providers but focus on long-term temporal outcomes of changing cohorts of patients and providers in a system. The first seeks to study treatment effects of an intervention on individuals while the latter seeks to study outcome effects of an intervention on system processes independent of individual effects. System-wide studies use both traditional statistical analysis, such as time-to-event models, and SPC methodologies, such as process control charts, to calibrate system outcome and variability over time before intervention and to measure outcome and variability during and after intervention to provide rigorous and statistically quantifiable quality measures that are generalizable across environments.

Besides application of SPC methodologies to assess QI programs, there are other critical key success factors in QI initiatives that were present in this study and in similar long-term QI initiatives on insulin intensification using a collaborative team care model (26). These factors include executive and clinical leadership commitment for QI, staff QI support and training programs, identifying improvement opportunities, formation of multidisciplinary teams, workforce support for process analysis and redesign, and personnel policies supporting and motivating staff for QI.

While the goal of improving care quality is widely shared, transitions to value-based payment models necessitate making both short- and long-term economic cases for improvement. This study was funded by a grant to Tulane from the ADA. Indirect support was obtained from industry for continuing medical education. The study did not develop or track short-term incremental costs associated with a preexisting QI team. Most institutions and large clinical systems have such QI teams in place, and the interventions used in our study can be applied at a modest incremental cost. The long-term benefits of this and another Diabetes INSIDE program were extrapolated from the clinical outcomes onto a 5-year risk of complications using a validated risk prevention model (32) showing an 18% reduction in all-cause mortality and a 22% reduction in cardiovascular disease death (33). Nuckols et al. (25) systematically reviewed the short- and long-term economic case for QI interventions in diabetes and found a “fair-to-good value relative to usual care.”

Improving outcomes for all patients with diabetes will take more than improving awareness, education, technology, and therapies. Population health management is needed across patients, providers, health care systems, communities, regions, and nations. QI initiatives like this and similar studies (26) are an effective strategy at improving population health locally and nationally, if widely adopted.

Acknowledgments. The authors acknowledge the valuable contributions of QI team members Venkat Ayinapudi, Markus Cormier, Cecilia Hatfield, Todd Nielsen, and Wanda Pellet from Tulane University Medical Group; Kerri Dotson, A. Jenine Dabon, Dr. Dragana Lovre, Dr. Rade Pejic, and Dr. Kristen Valliant from Tulane University School of Medicine; Rica Fontenette, Suzette Jackal, Debbie Ragas, Leslie Thomas, and Susan Villalobos from Tulane Medical Center; and Dr. Grace Thacker from Walgreens.

Funding. This study was supported by unrestricted educational grants from Eli Lilly and Company, Novo Nordisk, and Sanofi. The findings and conclusions in this study are those of the authors and do not necessarily represent the views of the funders.

Duality of Interest. G.L.’s spouse is an employee of Janssen Pharmaceuticals. V.A.F. has received research support (to Tulane) from Bayer and Boehringer Ingelheim; has received honoraria for consulting and lectures from Takeda, Novo Nordisk, Sanofi, Eli Lilly, Abbott, AstraZeneca, and Asahi; and has stock options in Microbiome Technologies, Insulin Algorithms, and BRAVO4Health. None of these relate directly to the study. No other potential conflicts of interest relevant to this article were reported.

Author Contributions. R.E.F. taught the QI training, performed the data analysis, advised the Tulane QI team, and wrote the manuscript. T.S.H. co-led the QI team, helped design and implement the QI interventions, and reviewed the manuscript. L.L. was the program manager for the QI team’s work with the Clinical and Information Technology departments and reviewed the manuscript. E.C.F. coordinated and advised the Tulane QI team and reviewed the manuscript. G.L. initiated and helped develop and plan the ADA QI program and reviewed the manuscript. V.A.F. co-led the QI team, helped design the QI interventions, and reviewed the manuscript. R.E.F. is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Prior Presentation. Parts of this study were presented as a poster at the 79th Scientific Sessions of the American Diabetes Association, San Francisco, CA, 7–11 June 2019.

1.
Stone
MA
,
Charpentier
G
,
Doggen
K
, et al.;
GUIDANCE Study Group
.
Quality of care of people with type 2 diabetes in eight European countries: findings from the Guideline Adherence to Enhance Care (GUIDANCE) study
.
Diabetes Care
2013
;
36
:
2628
2638
2.
de Pablos-Velasco
P
,
Parhofer
KG
,
Bradley
C
, et al
.
Current level of glycaemic control and its associated factors in patients with type 2 diabetes across Europe: data from the PANORAMA study
.
Clin Endocrinol (Oxf)
2014
;
80
:
47
56
3.
Khunti
K
,
Gomes
MB
,
Pocock
S
, et al
.
Therapeutic inertia in the treatment of hyperglycaemia in patients with type 2 diabetes: a systematic review
.
Diabetes Obes Metab
2018
;
20
:
427
437
4.
Dai
M
,
Peabody
MR
,
Peterson
LE
,
Mainous
AG
.
Adherence to clinical guidelines for monitoring diabetes in primary care settings
.
Fam Med Community Health
2018
;
6
:
161
167
5.
Blonde
L
,
Aschner
P
,
Bailey
C
,
Ji
L
,
Leiter
LA
,
Matthaei
S
;
Global Partnership for Effective Diabetes Management
.
Gaps and barriers in the control of blood glucose in people with type 2 diabetes
.
Diab Vasc Dis Res
2017
;
14
:
172
183
6.
Donabedian
A
.
Evaluating the quality of medical care: 1966
.
Milbank Q
2005
;
83
:
691
729
7.
Berwick
D
,
Fox
DM
.
“Evaluating the quality of medical care”: Donabedian’s classic article 50 years later
.
Milbank Q
2016
;
94
:
237
241
8.
Batalden
P
,
Davidoff
F
,
Marshall
M
,
Bibby
J
,
Pink
C
.
So what? Now what? Exploring, understanding and using the epistemologies that inform the improvement of healthcare
.
BMJ Qual Saf
2011
;
20
(
Suppl. 1
):
i99
i105
9.
Institute for Healthcare Improvement
.
The breakthrough series: IHI’s collaborative model for achieving breakthrough improvement
.
Diabetes Spectr
2004
;
17
:
97
101
10.
Committee on Quality of Health Care in America, Institute of Medicine
.
Crossing the Quality Chasm: A New Health System for the 21st Century
.
Washington, D.C.
,
The National Academies Press
,
2001
, p.
360
11.
American Diabetes Association
.
Introduction: Standards of Medical Care in Diabetes—2019
.
Diabetes Care
2019
;
42
(
Suppl. 1
):
S1
S2
12.
American Society for Quality
.
The Define, Measure, Analyze, Improve, Control (DMAIC) Process
[Internet]. Available from https://asq.org/quality-resources/dmaic. Accessed 4 September 2019
13.
Liptak
G
,
Furman
E
,
Furman
R
.
Diabetes INSIDE: a national quality improvement initiative guiding large health systems to improve diabetes care
. Diabetes
2016
;
65
(
Suppl. 1
):
A169
A220
14.
Lim
LL
,
Lau
ESH
,
Kong
APS
, et al
.
Aspects of multicomponent integrated care promote sustained improvement in surrogate clinical outcomes: a systematic review and meta-analysis
.
Diabetes Care
2018
;
41
:
1312
1320
15.
Ebix Inc
.
More than Just Patient Education: eClinical Works and A.D.A.M. SmartCare
[Internet]. Available from https://www.adam.com/ecw. Accessed 4 September 2019
16.
R Core Team
.
R: A language and environment for statistical computing
[Internet], 2017. Vienna, Austria, R Foundation for Statistical Computing. Available from https://www.R-project.org/. Accessed 4 September 2019
17.
Andersen
PK
,
Borgan
O
,
Gill
RD
,
Keiding
N
.
Statistical Models Based on Counting Processes (Springer Series in Statistics)
[Internet], 1993. New York, Springer-Verlag. Available from https://www.springer.com/us/book/9780387945194. Accessed 4 September 2019
18.
Therneau
TM
.
A Package for Survival Analysis in S
[Internet], 2015. Available from https://CRAN.R-project.org/package=survival. Accessed 4 September 2019
19.
Kleinbaum
DG
,
Klein
M
.
Survival Analysis: A Self-Learning Text, Third Edition (Statistics for Biology and Health)
[Internet], 2012. New York, Springer-Verlag. Available from www.springer.com/us/book/9781441966452. Accessed 4 September 2019
20.
Grambsch
PM
,
Therneau
TM
.
Proportional hazards tests and diagnostics based on weighted residuals
.
Biometrika
1994
;
81
:
515
526
21.
Montgomery
DC
.
Introduction to Statistical Quality Control
.
Hoboken, NJ
,
Wiley
,
2013
22.
American Diabetes Association
.
6. Glycemic targets: Standards of Medical Care in Diabetes—2019
.
Diabetes Care
2019
;
42
(
Suppl. 1
):
S61
S70
23.
Driskell
OJ
,
Holland
D
,
Waldron
JL
, et al
.
Reduced testing frequency for glycated hemoglobin, HbA1c, is associated with deteriorating diabetes control
.
Diabetes Care
2014
;
37
:
2731
2737
24.
Tricco
AC
,
Ivers
NM
,
Grimshaw
JM
, et al
.
Effectiveness of quality improvement strategies on the management of diabetes: a systematic review and meta-analysis
.
Lancet
2012
;
379
:
2252
2261
25.
Nuckols
TK
,
Keeler
E
,
Anderson
LJ
, et al
.
Economic evaluation of quality improvement interventions designed to improve glycemic control in diabetes: a systematic review and weighted regression analysis
.
Diabetes Care
2018
;
41
:
985
993
26.
Eisenstat
SA
,
Chang
Y
,
Porneala
BC
, et al
.
Development and implementation of a collaborative team care model for effective insulin use in an academic medical center primary care network
.
Am J Med Qual
2017
;
32
:
397
405
27.
Lyon
AW
,
Higgins
T
,
Wesenberg
JC
,
Tran
DV
,
Cembrowski
GS
.
Variation in the frequency of hemoglobin A1c (HbA1c) testing: population studies used to assess compliance with clinical practice guidelines and use of HbA1c to screen for diabetes
.
J Diabetes Sci Technol
2009
;
3
:
411
417
28.
Goodney
PP
,
Newhall
KA
,
Bekelis
K
, et al
.
Consistency of hemoglobin A1c testing and cardiovascular outcomes in Medicare patients with diabetes
.
J Am Heart Assoc
2016
;
5
:
e0035661
29.
Deichmann
RE
,
Castello
E
,
Horswell
R
,
Friday
KE
.
Improvements in diabetic care as measured by HbA1c after a physician education project
.
Diabetes Care
1999
;
22
:
1612
1616
30.
American Diabetes Association
.
1. Improving care and promoting health in populations: Standards of Medical Care in Diabetes—2019
.
Diabetes Care
2019
;
42
(
Suppl. 1
):
S7
S12
31.
Lipska
KJ
,
Yao
X
,
Herrin
J
, et al
.
Trends in drug utilization, glycemic control, and rates of severe hypoglycemia, 2006–2013
.
Diabetes Care
2017
;
40
:
468
475
32.
Shao
H
,
Fonseca
V
,
Stoecker
C
,
Liu
S
,
Shi
L
.
Novel risk engine for diabetes progression and mortality in USA: Building, Relating, Assessing, and Validating Outcomes (BRAVO)
.
Pharmacoeconomics
2018
;
36
:
1125
1134
33.
Shao
H
,
Shi
L
,
Furman
RE
,
Meneghini
L
,
Harlan
TS
,
Fonseca
V
.
174-LB: impact of quality improvement (QI) program on 5-year risk of diabetes-related complications
.
Diabetes
2019
;
68
(
Suppl. 1
):
174-LB
Readers may use this article as long as the work is properly cited, the use is educational and not for profit, and the work is not altered. More information is available at http://www.diabetesjournals.org/content/license.

Supplementary data