OBJECTIVE—Most quality improvement efforts for type 2 diabetes have neglected cardiovascular risk factors and are limited by a lack of information about generalizability across settings or persistence of effect over time.
RESEARCH DESIGN AND METHODS—We previously reported 6-month results of a controlled study of an intervention that improved cardiovascular risk factors for rural patients with type 2 diabetes. We subsequently provided the identical intervention to the control region after the main study was completed. The primary outcome was 10% improvement in systolic blood pressure, total cholesterol, or HbA1c. We compared the previously reported 6-month effect of the original intervention with the effect of the crossed-over intervention to the former control region and remeasured outcomes in the original intervention region 12 months later.
RESULTS—Our analysis included 200 original intervention and 181 crossed-over intervention subjects. The age of the population was 62.4 ± 12.4 years (mean ± SD), and 54.3% were women. A similar proportion of patients in the crossed-over intervention group achieved improvement in the primary composite outcome compared with the original intervention group (38 vs. 44%, respectively; P = 0.29). In adjusted analyses, we observed less improvement in blood pressure (adjusted odds ratio 0.40 [95% CI 0.17–0.75]) but greater improvements in total cholesterol (1.86 [0.93–3.7]) with the crossed-over intervention compared with the original intervention. We observed sustained improvements in total cholesterol and HbA1c levels in the original intervention group, whereas previous large gains in control of blood pressure diminished over time.
CONCLUSIONS—We found that our intervention was generalizable across settings, and its effect persisted over time. Nevertheless, without ongoing intervention or reinforcement, we noted some loss of the original benefits that had accrued. Future translational work should incorporate interventions such as ours into ongoing systems of rural care.
Despite high-quality evidence and widely endorsed clinical practice guidelines supporting the use of medications to decrease blood pressure, total cholesterol, and blood glucose (1–5), there is increasing recognition of treatment gaps in diabetes care (6–12). Because most morbidity and mortality in type 2 diabetes are related to macrovascular complications (13), interventions should aim to improve total cardiovascular risk (14) and not just glycemic control. Because rural patients may face a lack of local resources or restricted access to specialists and multidisciplinary clinics, they may be at particular risk for poorer quality of care (10, 12, 15, 16).
Improvements in diabetes management using multifaceted quality improvement interventions in primary care settings have been previously reported (17–19). Most of these studies, however, have been small, poorly controlled, or focused only on glycemic control (17). Controlled trials of multifaceted interventions to improve the quality of diabetes care in real-world settings are required, and beyond simple efficacy or effectiveness, these studies should address issues of generalizability and persistence (20).
We previously reported the results of a before/after study with concurrent control subjects of a multifaceted outreach intervention aimed at rural primary care providers and designed to improve quality of care for their patients with type 2 diabetes (18–19). Among patients in the intervention region, we observed a trend toward improvement in the primary composite outcome of blood pressure, total cholesterol, or HbA1c, but only improvements in control of blood pressure achieved conventional statistical significance compared with the control region (19). The intervention was associated with an increase in new medication starts for each of the targeted clinical outcomes (19), and we also observed improvements in patient-reported outcomes such as health-related quality of life and satisfaction with care (21). Our original study was limited, as most studies in this area are, by a relatively short follow-up period and the examination of only one intervention region (19). We could not answer the questions of whether our intervention was generalizable across health care settings and whether the effect of the intervention persisted longer than 6 months.
Recognizing these potential limitations of our original study, we conducted a prespecified “crossover” of the previously reported intervention to the original control region and completed an extended 18-month follow-up of all patients enrolled in the original study to address the question of persistence (18).
RESEARCH DESIGN AND METHODS
We originally conducted a before/after study with concurrent control subjects to evaluate our aforementioned intervention (18, 19). In summary, two generally comparable and geographically adjacent rural health regions in Northern Alberta, Canada, were selected and randomly allocated to intervention or control. Both regions were approximately a 6-h drive from the nearest secondary or tertiary care referral centers. Subjects were enrolled by local coordinators/nurse educators if they had type 2 diabetes, gave informed consent, and had sufficient English literacy to answer questionnaires. We excluded those who were unable or unwilling to provide consent and those with foreshortened life expectancy. Ethics approval was provided by the Health Research Ethics Board, University of Alberta.
For the original study, one region received the intervention (original intervention) while the other served as the usual care control (original control). Patients were enrolled and follow-up was conducted 6 months after the intervention started. The identical intervention was then provided to the original control region (i.e., crossed-over intervention). All previously enrolled patients were asked to return for assessment of the primary outcome ∼18 months after their original baseline assessment; this time period was 6 months after the crossover intervention was completed.
Diabetes Outreach Van Enhancement (DOVE) intervention
In addition to usual care, the intervention consisted of exposing a region to a diabetes outreach service (18, 19). The intervention was aimed at health care providers (primarily physicians) in the region and not directly at patients themselves. The service consisted of a team of specialist physicians, nurse educators, dietitians, and pharmacists. The service traveled to the largest communities in the region over a 2- to 3-day period on a monthly basis for 6 months delivering targeted educational messages. Each specialist physician visited the intervention region twice, carrying the same primary message each time, emphasizing control of blood pressure, cholesterol, or glucose with an overall goal of improved vascular health (18, 19). Educational messages were delivered based on techniques of group academic detailing (22, 23). The core of the intervention, delivered in identical fashion to both regions, was delivered by specialist physicians to small groups (i.e., two to six) of primary care physicians, discussing assessment and drug therapy with case studies and taking advantage of teachable moments with particularly difficult-to-treat patients identified by the local primary care physicians; the study pharmacist also met with all local primary care physicians for one-to-one reinforcement visits (traditional academic detailing).
Measurements
Our primary interest was “improvement” in the care of patients with diabetes; we operationalized this as ≥10% improvement in systolic blood pressure, total cholesterol, or HbA1c 6 months after exposure to the intervention (18, 19). We also evaluated changes in each separate component as prespecified secondary outcomes as well as initiation of new medications for control of blood pressure, total cholesterol, or HbA1c.
Baseline and follow-up data were collected by interviews, physical assessments, laboratory testing, and self-report questionnaires. Six trained study coordinators conducted interviews and collected information including demographics, histories, and detailed medication profiles. Standardized physical assessments were used to record weight, height, and blood pressure. Fasting blood samples were collected locally but analyzed centrally in one laboratory to determine HbA1c and cholesterol levels. Prespecified data collections occurred at baseline, 6 months, and 18 months after study entry.
Analysis
The primary analysis used a χ2 test of the proportion of subjects achieving the primary composite outcomes (i.e., 10% improvement in blood pressure, total cholesterol, or HbA1c); we calculated P values, odds ratios (ORs), and 95% CIs. For this follow-up study, we were interested in two comparisons: 1) the effect of the crossed-over intervention compared with the effect observed in the original intervention (i.e., a measure of generalizability) and 2) the effect of the original intervention 1 year after the original study was completed (i.e., a measure of persistence). Subjects with values already at or below all three clinical targets (blood pressure <130/80 mmHg, total cholesterol levels <5.0 mmol/l, and HbA1c <115% of upper limit of normal) were excluded from these analyses.
We were not able to adjust analyses for the possibility of provider-level “statistical clustering,” because we did not collect physician-specific data. In enlisting the participating regions, we agreed that we would not be auditing individual physician practices. Furthermore, the average cluster size was small to modest at best, with perhaps 5–10 patients per physician. For these reasons and because our goal was to change the regional provider culture about managing diabetes, we selected the patient as the unit of analysis and causal inference for all analyses.
To address potential imbalances between regions in important clinical and sociodemographic characteristics, multiple logistic regression was used. The dependent variable in this model was achievement of ≥10% improvement for primary composite outcome for each individual. Independent variables were intervention status; baseline blood pressure, total cholesterol, and HbA1c; and other statistically or clinically significant covariates (i.e., age, sex, duration of diabetes, use of target medications, marital status, and Aboriginal status). The same analytic approach was applied to each of the individual components of the primary outcome. All analyses were based on an intention-to-treat framework, whereby missing values at the end of follow-up were imputed with each patient’s baseline values.
RESULTS
We originally enrolled a total of 393 individuals with type 2 diabetes (210 intervention, 183 control) (19) and attempted follow-up on all patients. A total of 99 subjects (25%) dropped out of the study, died, or were lost to follow-up: 39 (19%) in the original intervention region and 60 (33%) in the crossed-over intervention (formerly the original control) region. Sociodemographic and clinical characteristics of the patients included in this analysis, stratified by region, are presented in Table 1.
Generalizability of the intervention effect across settings
In the former original control region, 6 months after we crossed over our multifaceted intervention, we found that 38% of subjects achieved ≥10% improvement in the primary study outcome compared with 44% improvement that we originally reported for our intervention (OR 0.80 [95% CI 0.53–1.21]; P = 0.29) (Fig. 1). In contrast to the original intervention (19), we observed less improvement in blood pressure control with the crossed-over intervention but noted greater improvements in total cholesterol level (Fig. 1). With the crossed-over intervention, 33 of 121 (27%) subjects with blood pressure >130/80 mmHg at baseline had ≥10% improvement, compared with 51 of 122 (42%) subjects in the original intervention (P = 0.02). On the other hand, the crossed-over intervention was associated with greater improvements in total cholesterol level than we had noted with the original intervention (Fig. 1). For total cholesterol, 30 of 162 (19%) of crossed-over intervention patients achieved ≥10% improvement versus 23 of 176 (13%) original intervention patients (P = 0.17 for difference). Changes in HbA1c were virtually identical between the crossed-over intervention and the original intervention groups (Fig. 1).
After we controlled for differences in demographic and clinical characteristics with multiple logistic regression, patients in the crossed-over intervention region were less likely to achieve improvements in blood pressure (adjusted OR 0.40 [95% CI 0.17–0.75]; P = 0.006) but seemed more likely to achieve improvements in total cholesterol level (1.86 [0.93–3.72]; P = 0.08) than those observed with the original intervention. There was no difference in improvements of glycemic control between the intervention periods (adjusted P = 0.98). Overall, there was a suggestion that subjects in the crossed-over group appeared somewhat less likely than those in the original intervention group to achieve improvements in the overall composite outcome after exposure to the diabetes outreach service (0.60 [0.36–1.08]; P = 0.06).
During the original study period, we observed new target medication starts for ∼25% of all subjects, with absolute increases seen for all three target medications in the original intervention group compared with the original control group (19). We found increases in new target medication starts with the crossed-over intervention to be similar in magnitude to those seen with the original intervention, although blood pressure medication starts were still more likely among patients during the original intervention period (Table 2). Although an uncontrolled observation, exposure to the intervention at any time period seemed to be associated with more aggressive management of each of the three targeted clinical indicators (Table 3). A greater proportion of patients above targets were treated overall, and more patients were treated with a combination of two or three drug therapies (Table 3).
Persistence of the intervention effect over time
With respect to persistence of the effect of the original intervention, we observed further improvements in glycemic and cholesterol control in the year after the original intervention was completed (Fig. 1). Improvements in HbA1c were similar, with 22% of subjects in the extended follow-up period compared with 18% in the original intervention (P = 0.49), whereas significantly greater improvements were observed in total cholesterol level (22 vs. 13%; P = 0.02). On the other hand, we observed a decay of the earlier large improvements for blood pressure control, with only 29% improved at 18 months compared with 42% improved at 6 months (P = 0.003). Nonetheless, improvement in blood pressure control was still better in the intervention region after our study than it had been at baseline (Fig. 1).
CONCLUSIONS
Implementing evidence-based practice to improve quality of care in diabetes is difficult. Advances have been made in translating knowledge into practice in the field of diabetes, but several challenges remain. In a recent commentary from the Diabetes Mellitus Interagency Coordinating Committee, Garfield et al. (20) identified priorities for translational research in diabetes, including the conduct of multifaceted, population-based intervention programs, with attention to the generalizability and sustainability or persistence of such interventions. Narayan et al. (24) echoed these recommendations, emphasizing that these complex interventions need to be formally tested in randomized or quasi-experimental practical trials.
We previously reported that our multifaceted intervention led to a 19% relative improvement in our composite diabetes quality-of-care indicator over a period of 6 months (19). In that study, we noted more frequent medication starts for the control of blood pressure, cholesterol, and glucose in the intervention region compared with those in the control region (19). Although exposure to the diabetes outreach service did not result in significant changes in cholesterol or glucose levels during the short 6-month follow-up period, we did note a statistically significant and clinically important improvement in blood pressure control. As a potential explanation for these results, we suggested that this represented the longer lag times between medication starts and the clinical effects on cholesterol-lowering or glucose-lowering compared with blood pressure changes. Nevertheless, we recognized that questions would remain regarding the potential generalizability and sustainability of our findings.
Therefore, we planned, a priori, a crossover of the intervention to the original control region and an 18-month follow-up of all patients originally enrolled in the study (18). We had speculated that a longer follow-up period might reveal sustained improvements in clinical outcomes and further increased use of targeted medications. On the other hand, in the absence of ongoing reinforcement or continued interventions, it might be expected that some decay in the short-term enhancements in quality of care might result. To some degree, we observed both situations over our longer-term follow-up study.
After the intervention was crossed over to the former control region, we observed similar improvements in the primary composite outcome and improvements in the individual components of HbA1c and total cholesterol level. Improvements in these clinical indicators were generally accomplished through more aggressive medication management. In fact, the need for use of combination therapies was a constant theme of our educational messages (18, 19). There were, however, some differences in the patterns of improvement, although overall quality of diabetes care (as we defined it) improved. One region adopted hypertension messages more readily, whereas another region adopted messages related to cholesterol more readily. Regardless of the specific paths taken to improvement, our findings suggest that the intervention is, to some degree, generalizable across settings.
In the longer-term follow-up of patients in the original intervention region, we observed consistently improving control of HbA1c and total cholesterol level, perhaps reflecting the physiologic delay after the new medication starts for these two elements of metabolic control in diabetes we previously observed (19). Furthermore, although blood pressure control in the original intervention region was still improved, we did note a decrease in the originally observed improvements (19). This suggests that the persistence of quality improvements may be limited in the absence of ongoing intervention. It is important, therefore, to develop strategies for the sustainability and integration of effective quality improvement interventions, tested in similar pilot fashion, into dynamic health care systems. This is the very crux of translational research and requires commitment of health researchers to follow through on the translation activities, and health care decision makers, at both clinical and policy levels, to heed the evidence by participating in and funding the proven effective strategies.
Although there were multiple components to our intervention, the primary method was delivery of evidence-based messages by specialist physicians using group academic detailing. We recruited three locally well-known specialists in hypertension, lipids, and diabetes and instructed them in methods of detailing, helped them develop educational messages and detailing materials, and then had them travel to rural regions and interact in person with small groups of rural physicians. The same three specialists provided the same intervention in the follow-up period, and no subsequent intervention was, to our knowledge, provided to the original intervention region. It is possible that the message was confounded by the messenger, such that physicians in one region were more receptive to the educational influences from one particular specialist than the other. Therefore, one region’s physicians may have more avidly adopted messages related to hypertension, whereas the other’s physicians were more influenced with respect to messages related to total cholesterol. Unfortunately, there is no way for us to examine these hypotheses further within the current study design.
Our study has all of the strengths of a valid controlled intervention trial and the limitations shared by all nonrandomized studies. We recognize a differential drop-out rate in our study, reflective of the pattern observed in the original 6-month follow-up (19) but increased with the extended follow-up. We applied the conventional intent-to-treat strategy in our analysis. With this strategy, whereby the outcome measurements are assumed to be unchanged from baseline, the larger proportion of subjects in the crossed-over intervention conservatively biases any comparisons toward the null. Only one death (in the crossover intervention group) was noted among all enrolled subjects.
Interventions studied at the level of clinics or regions might be evaluated using a cluster design to account for lack of statistical independence among individual patients seen by the same provider (23). Because we agreed with the regions that we would in no way undertake an audit of individual physicians’ practices at study inception, we did not collect any physician-specific information and, indeed, gathered all of our own physical measures and outcomes data without resort to chart review. This necessity precluded analyses that could control for potential clustering (e.g., using hierarchical regression or generalized estimating equations methods) (23).
Nevertheless, our study overcomes many of the limitations of previous research on quality improvement interventions for type 2 diabetes (17–19, 20). We feel that the combined data from the original DOVE Study and this follow-up report comply with the standards for reporting of nonrandomized studies of public health interventions suggested by the TREND Group (25). Furthermore, there are very few translational studies that even attempt to examine questions related to either generalizability of an intervention effect on persistence or sustainability over the longer term (20, 24). The fact that we observed similar patterns of improvement when the intervention was crossed over to the former original control region lends further support to the premise that an outreach service does result in improvements in the quality of diabetes care.
In conclusion, we believe that a multifaceted diabetes outreach service (that included specialist-to-generalist group academic detailing) has the potential to improve the quality of diabetes care for patients in isolated rural communities. We have now, to some degree, demonstrated that the clinical benefits of this outreach intervention are generalizable across time and place and suggest that our findings are of sufficient validity for this intervention to be adapted and implemented in a wide variety of rural settings.
Percentage of subjects achieving improvement (i.e., ≥10% change) in clinical outcomes by original (19) and crossover intervention study periods.
Percentage of subjects achieving improvement (i.e., ≥10% change) in clinical outcomes by original (19) and crossover intervention study periods.
Patient characteristics stratified by region
| Original intervention | Crossed-over intervention | |
|---|---|---|
| n | 200 | 181 |
| Age (years) | 63.3 ± 12.3 | 61.3 ± 12.4 |
| Sex (% men) | 51.5 | 61.6 |
| Married | 70.7 | 66.1 |
| Aboriginal (%) | 10.2 | 47.8 |
| Completed high school (%) | 33.4 | 29.8 |
| Duration of diabetes (years) | 7.6 ± 7.8 | 9.2 ± 9.2 |
| Visited diabetes education clinic? (% yes) | 66.7 | 49.7 |
| Years since last visit | 3.4 ± 4.3 | 4.1 ± 4.7 |
| BMI (kg/m2) | 33.5 ± 7.1 | 32.3 ± 6.2 |
| Systolic blood pressure (mmHg) | 131.3 ± 19.0 | 133.1 ± 18.9 |
| Diastolic blood pressure (mmHg)* | 72.8 ± 11.5 | 79.3 ± 10.4 |
| Total cholesterol (mmol/l) | 4.94 ± 0.89 | 5.06 ± 0.93 |
| HbA1c (%)* | 7.24 ± 1.48 | 7.75 ± 1.78 |
| Original intervention | Crossed-over intervention | |
|---|---|---|
| n | 200 | 181 |
| Age (years) | 63.3 ± 12.3 | 61.3 ± 12.4 |
| Sex (% men) | 51.5 | 61.6 |
| Married | 70.7 | 66.1 |
| Aboriginal (%) | 10.2 | 47.8 |
| Completed high school (%) | 33.4 | 29.8 |
| Duration of diabetes (years) | 7.6 ± 7.8 | 9.2 ± 9.2 |
| Visited diabetes education clinic? (% yes) | 66.7 | 49.7 |
| Years since last visit | 3.4 ± 4.3 | 4.1 ± 4.7 |
| BMI (kg/m2) | 33.5 ± 7.1 | 32.3 ± 6.2 |
| Systolic blood pressure (mmHg) | 131.3 ± 19.0 | 133.1 ± 18.9 |
| Diastolic blood pressure (mmHg)* | 72.8 ± 11.5 | 79.3 ± 10.4 |
| Total cholesterol (mmol/l) | 4.94 ± 0.89 | 5.06 ± 0.93 |
| HbA1c (%)* | 7.24 ± 1.48 | 7.75 ± 1.78 |
Data are means ± SD, unless otherwise indicated. *P < 0.01.
New medication starts in 6-month follow-up for individuals above target at baseline
| Original intervention | Crossed-over intervention | P value | |
|---|---|---|---|
| Any target medication | 53 (27) | 48 (27) | 0.99 |
| n | 200 | 181 | |
| Blood pressure lowering | 24 (20) | 19 (16) | 0.42 |
| n | 121 | 122 | |
| Total cholesterol lowering | 14 (8) | 19 (12) | 0.24 |
| n | 176 | 162 | |
| Glucose lowering | 17 (18) | 19 (16) | 0.78 |
| n | 97 | 118 |
| Original intervention | Crossed-over intervention | P value | |
|---|---|---|---|
| Any target medication | 53 (27) | 48 (27) | 0.99 |
| n | 200 | 181 | |
| Blood pressure lowering | 24 (20) | 19 (16) | 0.42 |
| n | 121 | 122 | |
| Total cholesterol lowering | 14 (8) | 19 (12) | 0.24 |
| n | 176 | 162 | |
| Glucose lowering | 17 (18) | 19 (16) | 0.78 |
| n | 97 | 118 |
Data are n (%).
Distribution of target medications at final visits for individuals not at target at baseline (i.e., 6 months before)
| No. of medications | HbA1c | Blood pressure* | Total cholesterol | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Original | Crossover | Original | Crossover | Original | Crossover | ||||
| 0 | 6.2 | 5.9 | 19.7 | 32.2 | 73.3 | 75.9 | |||
| 1 | 29.9 | 33.9 | 28.7 | 28.9 | 24.4 | 23.5 | |||
| 2 | 46.4 | 41.5 | 27.9 | 24.0 | 2.3 | 0.6 | |||
| ≥3 | 17.5 | 18.6 | 23.7 | 14.9 | — | — | |||
| No. of medications | HbA1c | Blood pressure* | Total cholesterol | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Original | Crossover | Original | Crossover | Original | Crossover | ||||
| 0 | 6.2 | 5.9 | 19.7 | 32.2 | 73.3 | 75.9 | |||
| 1 | 29.9 | 33.9 | 28.7 | 28.9 | 24.4 | 23.5 | |||
| 2 | 46.4 | 41.5 | 27.9 | 24.0 | 2.3 | 0.6 | |||
| ≥3 | 17.5 | 18.6 | 23.7 | 14.9 | — | — | |||
Data are percent.
For the blood pressure distribution, P = 0.038; HbA1c and total cholesterol treatment patterns were not statistically different.
Article Information
Financial support for this study was provided by the Institute of Health Economics, Caritas Health Group, and the Canadian Diabetes Association. This work was also supported by a New Emerging Team (NET) grant to the Alliance for Canadian Health Outcomes Research in Diabetes (ACHORD). The ACHORD NET grant is sponsored by the Canadian Diabetes Association; the Heart and Stroke Foundation of Canada; The Kidney Foundation of Canada; the CIHR–Institute of Nutrition, Metabolism and Diabetes; and the CIHR–Institute of Circulatory and Respiratory Health.
S.R.M. is a Population Health Investigator with the Alberta Heritage Foundation for Medical Research (AHFMR) and a New Investigator of the Canadian Institutes of Health Research. D.T.E. holds a Full-Time Studentship in Health Research with AHFMR. J.A.J. holds a Canada Research Chair in Diabetes Health Outcomes and is a Health Scholar with AHFMR.
We thank the health regions that participated in this study, the local study coordinators who identified and tracked study subjects, and the subjects themselves. We also thank Lisa Guirguis for her efforts in setting up the DOVE and DOVE follow-up studies and EPICORE (Epidemiology Coordinating and Research), Division of Cardiology, and University of Alberta for support with data management.
References
T.K.L. has received honoraria for speaking engagements from Servier Canada and GlaxoSmithKline.
A table elsewhere in this issue shows conventional and Système International (SI) units and conversion factors for many substances.
