Improving Diabetes Care in Midwest Community Health Centers With the Health Disparities Collaborative

In 1998, the Bureau of Primary Health Care invited federally funded community health centers to apply to be part of a Diabetes Collaborative quality improvement process. The initiative was national, but much of the implementation was performed at a regional level. In the Midwest, 22 health centers applied. A committee of clinicians, administrators, and representatives of the BPHC selected 20 centers based upon criteria including an adequate number of patients with diabetes, sufficient computer resources, electronic mail capability, and commitment of leadership. Each health center was expected to form a diabetes quality improvement team that would meet regularly with the support of senior administrative leadership. Each center would also create a registry of patients with diabetes to help track clinical care. One center dropped out at the beginning of the project, leaving nineteen centers.

A rapid change process called Plan, Do, Study, Act (PDSA) was introduced into each health center. This model, developed by the Institute for Healthcare Improvement, adapts elements of continuous quality improvement into a process designed to improve the quality of care at an accelerated pace. Following the establishment of a major aim for improving the quality of care, the elements of this process are as follows: 1) Plan—define an intervention to help the center achieve the major aim. The key element to planning involves the dissection of the intervention into small, measurable, and accomplishable steps; 2) Do—implement the intervention on a small scale; 3) Study—analyze the effects of the intervention; and 4) Act—based on the study data, revise the intervention. The emphasis is on rapid PDSA cycles that have quick turnaround times. Thus, the number of patients upon which an individual PDSA cycle is performed may be too small for change to reach statistical significance, but the goal is to have enough patients to determine practically if the intervention is working or requires revision.

The MacColl Chronic Care Model aims to create practical, supportive interactions between an informed, activated patient and a proactive, prepared clinical team. The model posits that multiple dimensions must be addressed if care is to be improved. The health system and organization of care must be improved, and community resources must be tapped. Specific targets for quality improvement are patient self-management, delivery system redesign, decision support, clinical information systems, leadership and health system organization, and community outreach.

The Institute for Healthcare Improvement provided initial instruction at a national learning session, and then regional cluster coordinators and institute staff (0.4 full-time equivalent) assisted through telephone conference calls, a computer listserve, feedback on required monthly progress reports, and three regional meetings. Monthly progress reports from the centers were expected to include aims, descriptions of rapid PDSA cycles, and reports of adherence to selected diabetes quality-of-care standards. At the regional learning sessions, team members and administrators from all 19 health centers met to learn quality-improvement techniques and share lessons among themselves.

The BPHC asked all health centers to perform at least two HbA_1c tests at least 3 months apart over the year for 90% of their target population. Health centers were also asked to pick other specific goals of their own choice as targets for quality improvement. Health centers were asked to start with a subset of their center’s population and then spread the intervention to other patients. By the end of the year, most health centers had targeted all of their diabetic patients at their given site.

We analyzed a variety of data for triangulation and increased validity of findings. We concentrated on process and outcome measures along both clinical and organizational lines. In a brief 1-year evaluation period, it may be difficult to see significant improvement in outcomes. However, the collaborative could be considered successful if we found improvement in intermediary processes expected to lead to better outcomes. The University of Chicago Institutional Review Board approved the study, and informed consent was obtained.

We used a preexisting chart review instrument developed by a team from The University of Chicago and the MidWest Clinicians’ Network to describe patient demographics and evaluate process-of-care measures based on the clinical recommendations of the American Diabetes Association (20). The chart review instrument was accompanied by a codebook, and each center was instructed in the chart abstraction process. Each center was asked to identify patients with diabetes through administrative records and International Classification of Diseases, Clinical Modification, 9th edition diagnostic code 250.x or patient registries (21), and to then perform chart review on 80 patients with diabetes (confirmed by chart review), aged 18–75 years and chosen by random number generator, or on all diabetic patients aged 18–75 years if their center had fewer than 80. Pregnant women were excluded. Each center abstracted data from the year before the collaborative (1998) and the year of the collaborative (1999). When possible, the same patients’ charts were abstracted in 1998 and 1999. If a patient analyzed in 1998 was no longer seen at the health center in 1999, then another randomly chosen diabetic patient’s chart was abstracted in 1999 so that close to 80 patients per center per year were analyzed. The average number of chart audits per center was 69 in 1998 and 79 in 1999. On average, 72% of the patients whose charts were audited in 1998 were followed up in 1999. To check on the reliability of the chart review, a random 5% sample of charts was reaudited by the health centers for the major American Diabetes Association process-of-care variables. The overall agreement of items was 84% with an average κ statistic of 0.65 (22).

Toward the end of the first year of the collaborative, we mailed a survey to members (n = 106) of each health center’s diabetes collaborative team. Nonrespondents were mailed two more copies of the survey and were then telephoned with a subset of the questions if they still had not responded. The survey addressed overall evaluation of the collaborative, quality improvement models and tools, education and assistance, communication, team functioning (23), diffusion and continuation of activities, time, costs, administrative support, and demographic information. Questions developed by the research team generally were rated on a five-point Likert-type scale.

We conducted semistructured telephone interviews with team leaders and team members, inquiring about the nature of the intervention, program implementation at each health center, overall assessment, facilitators and challenges to the intervention, and spread of the initiative. Team leaders were interviewed twice (in August 1999 and April/May 2000), and random team members were interviewed once (in April/May 2000). We also conducted similar interviews during the latter period with at least one randomly selected nonteam member at each health center in order to gauge the awareness of the intervention by other clinic staff and its diffusion throughout the health center. Interviews were audiotaped.

We performed descriptive statistics on patient demographic characteristics and compared the rates at which the processes-of-care standards were met from 1998 to 1999. The primary measures of the quality of care were eight major process measures and one outcome measure: 1) at least one measurement of HbA_1c, 2) two measurements of HbA_1c at least 3 months apart, 3) dilated eye examination, 4) diet intervention, 5) dental care, 6) foot care or foot education, 7) lipid assessment, 8) urine microalbumin assessment, and 9) the absolute value of HbA_1c (20).

The unit of analysis was the measure in a given year for an individual patient, so that individual measures were nested within patients and patients were nested within health centers. To incorporate correlation between measures on the same patient and between patients in the same health center, we used hierarchical regression (24–26). For HbA_1c, which was approximately normally distributed, we fit hierarchical linear models (HLMs) with patient and health center as random effects, using HLM statistical software, version 5. The eight process measures were dichotomous variables, which indicated whether a particular process of care had been performed, and were analyzed in HLM using hierarchical logistic regression, again with patient and health center as random effects. We performed two sets of outcome analyses. In the primary analyses, only patients present in both 1998 and 1999 are included. In the second set, patients present in either 1998 or 1999 are analyzed.

Survey results were tabulated by conceptual domain. Selected items, such as usefulness and burden of patient registry were also cross-tabulated. Both summary scales and individual items were included. The summary scales represent selected conceptual domains and include frequency and usefulness of communication between the team and collaborative, team interpersonal relations, team efficacy, usefulness of the Chronic Care Model components, intent to continue collaborative activities, and costs of participation in terms of time and effort. These multi-item scales were constructed as equally weighted averages. Items were reverse coded as needed to conform to a positive versus negative response scale. Reliability analysis indicated good internal consistency of the scales (Cronbach’s α 0.65–0.92) (27). To enhance comparability to individual items, summary scales were recoded to correspond to the original one- to five-point Likert scale using midpoints as cutpoints between levels (e.g., 4.5 as the cutpoint between “agree” and “strongly agree”).

Seventy-nine team members returned surveys, for a response rate of 75%. All 19 participating community health centers had three or more respondents (range 3–8, median 5). Ninety-three (88%) team members either returned a written questionnaire or answered the telephone survey. After verifying that the telephone responses were consistent with responses on the larger questionnaire, the data from the two sources were pooled.

Two investigators independently listened to each audiotape or read each transcription to ensure the validity and reliability of the theme analysis.

The patient demographic characteristics in 1998 and 1999 from the chart review were almost identical, differing by ≤3% in any category of any variable. Approximately two-thirds (64%) were women, one-third (1998: 33%; 1999: 32%) were African American, and 22% were Hispanic. About half (47 and 46%, respectively) were between 51 and 65 years of age, and one-quarter (24 and 22%) were 41–50 years old. About 90% (88 and 90%) had type 2 diabetes, and approximately one-third (34 and 33%) were taking insulin or insulin and an oral agent. One-fourth (24 and 23%) had Medicare insurance, one-fourth (26 and 24%) had Medicaid insurance, and about one-quarter (27 and 28%) were self-pay, usually on a sliding-fee schedule. Thirty-seven percent of the health centers were rural. On average, each center had 398 patients with diabetes (range 74–987).

Most health centers performed at least 30 different interventions over the course of the year. Examples of frequently performed interventions for each element of the Chronic Care Model are below.

• Community: collaborate with community organizations for health center’s population (89%).

• Patient self-management support: self-management tool or goal sheet to track a patient’s progress (63%).

• Delivery system redesign: group cluster visits (63%).

• Decision support: diabetes flow sheet (100%).

• Computer information systems: used patient registry to follow up on examination and laboratory data (37%).

The proportions of patients for whom each of the major processes of care were performed increased significantly from 1998 to 1999, as determined by chart review (Table 1). Rates of HbA_1c measurement, eye examination referral, dietary counseling, foot examination, dental referral, lipid assessment, and urine microalbumin assessment all improved, with ORs ranging from 1.28 to 2.92 for patients present in both 1998 and 1999 chart reviews. The mean value of HbA_1c decreased slightly (P = 0.09).

Survey respondents indicated a high level of support for the collaborative. About 95% of respondents agreed or strongly agreed that the collaborative was worth the effort and was successful (Table 2), and >80% intended to continue collaborative activities. Both the Chronic Care Model and the PDSA system were generally thought to be helpful, although 11% of respondents did not find the PDSA system useful. Teams generally worked well together. Regarding diffusion, 79% of respondents noted that patients or nonteam coworkers had commented on changes initiated by the team.

Areas that were most challenging concerned time and the related burden of data collection and report generation. Most respondents spent 1–5 h per week on collaborative activities, and 38% spent 6–10 h or more. Thirty-one percent of respondents, distributed among 15 of the 19 participating community health centers, reported inadequate time to work on collaborative activities. Most respondents agreed that they received support from their health center’s administration, but only ∼41% agreed that the administration had supplied release time from other duties for collaborative activities. A large majority of respondents considered the monthly reports and patient registry useful but also burdensome. Significant team member and clinic staff turnover was reported by about half of the respondents.

Overall, in interviews team leaders and team members were very proud of their accomplishments. Major perceived successes included improved diabetes care (e.g., lower HbA_1c, more self-management, better community linkages), better adherence to standards, more comprehensive care, increased awareness among patients and providers, development of patient registries and tracking systems, cluster clinics in which patients receive multiple aspects of care in one joint session, and networking with other health centers. The Chronic Care Model and rapid PDSA cycle models were useful, although many centers used the PDSA cycles qualitatively as opposed to quantitatively. Most nonteam members were aware of the collaborative intervention and thought that it had improved the system of diabetes care at their health centers.

Team leaders and members noted the need for more staff, time, and technical support, as well as for more supplies and educational materials for patients. About one-third of the centers expressed a desire for more senior support, and several centers reported that they wished there was someone in a position of authority to encourage provider adherence with the program. A frontline champion was usually needed in addition to senior management support.

Many centers initially had difficulties with developing their computerized patient registry systems. Some respondents also expressed a preference for clear direction, clear terminology, and many concrete examples. They wanted a clear structure, but valued their autonomy.

The BPHC’s Health Disparities Collaborative is the most ambitious effort to date to improve the quality of diabetes care in community health centers. Breakthrough Series concepts drawn from tenets of continuous quality improvement, chronic disease management, and collaborative learning have become increasingly popular in recent years, but evaluation of these techniques has been limited (18,28).

Wagner et al. (18) tested the Diabetes Collaborative approach in a group of 23 health care organizations, of which 7 were safety net providers. They noted that many teams reported improvements in several goals, including measurement of HbA_1c and actual HbA_1c level. Community health centers had some of the largest gains. Wagner et al. primarily used quality improvement data and methods for their study and suggested that more rigorous research and program evaluations needed to be done. Our study adds important information. It was designed to be a research evaluation and, thus, included rigorous data from multiple data sources and statistical techniques that adjusted for clustering of patients within centers. Our data also allowed us to describe some of the facilitators and barriers to the intervention in more detail. Our study includes a broader range of health centers and is therefore more generalizable to all types of community health centers.

Many health centers felt that they needed more resources to free staff time for the project. In addition, more resources may be needed to provide some services, such as dilated eye examinations. The delay in finding an efficient and common electronic patient registry impeded many centers from quickly identifying their population and starting desired interventions. The Chronic Care Model was universally thought to be useful. In comparison, the rapid PDSA methodology received less support, although it was still viewed favorably overall.

The proper balance of structure and autonomy needs to be found. The Breakthrough Series is a general model, but many health centers wanted a menu of concrete model interventions to choose from. By the end of the first year, different options were being presented, such as the learning sessions in which health centers could share model programs. Templates for basic infrastructural components, such as the patient registry, were provided. Buy-in from senior management was critical and variable, and subsequent Diabetes Collaborative efforts have facilitated this through explicit expectations and commitments from senior management and through training of both leaders and staff regarding this issue. A champion of the Breakthrough Series at each center was critical.

Our study has several limitations. Chart review examines charting practices that may not wholly represent the actual care provided. We may have underestimated the effect of the Breakthrough Series because we took a random sample of patients’ charts from each center. To the extent that a health center did not target all of its patients with diabetes at their site for interventions, random chart audit may underestimate the effects of the intervention. However, virtually all health centers reported that they had targeted all of their diabetic patients for interventions by the latter months of the initiative. An additional limitation existed because some of the patients targeted during the latter months of the year had a relatively short time to benefit from the intervention. Longer follow-up is needed to determine the full and lasting effects of the intervention. Also, not every health center targeted each of the individual American Diabetes Association standards for quality improvement. Thus, our analysis of pooled data from each health center about each quality standard may be a conservative estimate of the Breakthrough Series’ effect.

While our 75% survey response is high for a provider survey (29), nonrespondent bias is still possible. However, through telephone follow-up of nonrespondents to the written survey, we obtained an 88% response rate to several of the most important global evaluation questions of the Diabetes Collaborative. Given that the results of the telephone respondents were similar to those from the subsample of the 75% of providers answering the written questionnaire, we felt that bias was minimal.

In addition, our study is nonrandomized. Conceivably, secular changes unrelated to the intervention may have led to the improvement in diabetes care. However, this is unlikely as the Diabetes Collaborative was by far the dominant influence between the 2 years of data measurement.

Our study of the first year of the initiative indicates that the Breakthrough Series is a promising intervention to improve processes of diabetes care. In addition, since this first experience implementing a chronic disease collaborative in health centers has subsequently led to refinements in how the Breakthrough Series is organized and run, future collaboratives are even more likely to be successful. However, several important research questions remain. Whether quality improvement can be sustained beyond 1 year needs to be determined. Is it possible to sustain improvement without more funding for staff time? Can morale be maintained? Are continued collaborative activities necessary to ensure sustainability? If so, what types of activities and support are most critical? How much staff time needs to be freed for collaborative activities? Is it critical to free time for the team leader, the team members, or both parties?

In addition, it is unknown whether quality improvement efforts can survive major turnover in staff. How can the Breakthrough Series efforts be institutionalized so that health centers are not dependent on any one champion or few individuals? How does one create the culture and support for change and quality improvement?

As health centers broaden their quality improvement efforts, it will be important to determine how diabetes care can be integrated into an overall ongoing improvement process to avoid a segmented, “disease of the month” feel. Balancing attention and resources for diabetes with other worthy competing goals at the health centers will be challenging, and it will be important to determine if the collaborative intervention is cost-effective.

The BPHC’s Diabetes Breakthrough Series and wider 6-year Health Disparities Collaborative are bold interventions to improve the quality of care in health centers. Our initial evaluation of this effort in Midwest health centers indicates that the model, in conjunction with the enthusiasm and hard work of clinicians and administrators, has led to significant improvements in diabetes care in 1 year as assessed through chart review.

This project was supported by the Bureau of Primary Health Care, the Agency for Healthcare Research and Quality (R01 HS10479-01), and the National Institute of Diabetes and Digestive and Kidney Diseases Diabetes Research and Training Center (P60 DK20595). M.H.C. is a Robert Wood Johnson Foundation Generalist Physician Faculty Scholar.

Presented in part at the American Diabetes Association Annual Meeting, June 23, 2001, Philadelphia, Pennsylvania, and at the American Public Health Association Annual Meeting, October 22, 2001, Atlanta, Georgia.