OBJECTIVE—To assess the long-term effectiveness of a quality improvement program on care provided and patient outcomes in patients with diabetes.

RESEARCH DESIGN AND METHODS—A nonrandomized trial was performed with 312 patients with type 2 diabetes in the intervention group and 77 patients with type 2 diabetes in the reference group. The follow-up period was 42 months. The quality improvement program focused on improving both the provision of diabetes care and the patient outcomes. The program consisted of clinical practice guidelines, postgraduate education, audit and feedback, templates to register diabetes care, and a recall system. Data on the care provided were abstracted from medical records. Main outcomes on the provision of care were annual number of patient visits, blood pressure, and HbA1c and blood lipid levels. Main patient outcomes were blood pressure and HbA1c and blood lipid levels. Multilevel analysis was used to adjust for dependency between repeated observations within one patient and for clustering of patients within general practices.

RESULTS—Patients in the intervention group received care far more in accordance with the guidelines than patients in the reference group. Odds ratios ranged from 2.43 (95% CI 1.01–5.82) for the measurement of urine albumin to 12.08 (4.70–31.01) for the measurement of blood pressure. No beneficial effect was found on any patient outcome.

CONCLUSIONS—The quality improvement program improved the provision of diabetes care, but this was not accompanied by any effect on patient outcomes.

In recent decades, the treatment of patients with type 2 diabetes has shifted from specialist care to primary care (1,2). As a consequence, guidelines and disease management programs for primary care have been developed. Several studies have proved that passive dissemination of guidelines alone has little effect on improving clinical practice (3,4,5). According to two systematic reviews, a multifaceted intervention strategy seems to be a more effective method of improving clinical practice than the implementation of a single intervention strategy (6,7). For diabetes care, it has been proven that if regular, prompted recall and review of patients are guaranteed, primary care can achieve standards of care as good as or even better than hospital outpatient care, at least in the short term (8).

Because of conflicting results and methodological shortcomings of most studies, it is not clear which strategy is most effective in improving diabetes care. Only a few studies have assessed patient outcomes in addition to outcomes on the care provided. Insight into both outcomes on the care provided and patient outcomes is important (9), because the extent to which an intervention that is targeted at improving diabetes care has changed the process of care is masked when patient outcomes are measured without also measuring the process of care. On the other hand, if only the process of care is measured, it is not clear whether changes in the process of care produce any corresponding improvements in the health of patients.

In the Netherlands, patients with type 2 diabetes are mainly treated in general practice. In 1989, the Dutch College of General Practitioners formulated guidelines for the care of patients with type 2 diabetes. However, empirical data showed clear shortcomings in the diabetes care provided by general practitioners (GPs) (10). Therefore, a multifaceted quality improvement program, targeted at GPs, was introduced to improve the treatment of patients with type 2 diabetes. The aim of this study was to determine the long-term effectiveness of this quality improvement program on both the care provided to the patients and the patient outcomes.

From 1992 to 1997, a nonrandomized trial was performed in which GPs were allocated to an intervention group or a reference group. The follow-up period was 42 months. The study was approved by the Medical Ethics Committee of the Academic Hospital of the Vrije Universiteit in Amsterdam.

Study population

Recruitment of general practitioners.

The GPs recruited for the intervention group (IG) participated in a program developed by the Vrije Universiteit in Amsterdam to improve the quality of care for patients with chronic diseases in general practice. Their practices were located in an urbanized rural area in the eastern region of the Netherlands, as well as in Amsterdam and surrounding areas. The GPs recruited for the reference group (RG) came from the same regions and had comparable characteristics to those in the IG with regard to type of practice (single or group), size of practice, and previous education on diabetes care. GPs in the RG continued to provide usual care. A total of 22 GPs participated in the IG and 5 GPs participated in the RG.

Recruitment of patients.

All patients who were diagnosed with type 2 diabetes before 1992, according to the 1985 criteria of the World Health Organization (11), were identified. Patients were eligible for participation in the study if: 1) the GP was the main provider of diabetes care; 2) they were able to complete a questionnaire in the Dutch language; 3) informed consent was obtained. At baseline, 526 patients were identified in the intervention practices receiving GP care (66%); of these patients, 101 were excluded because they were unable to complete the questionnaire. Therefore, 425 patients met the inclusion criteria; 374 gave written informed consent.

In the RG, 119 patients with type 2 diabetes receiving GP care (68%) were identified; 14 patients were excluded because they were unable to complete the questionnaire, and 1 patient did not give informed consent. Consequently, 104 patients were included in the study.

In comparison with patients who were receiving specialist care, patients receiving GP care had a lower mean HbA1c (7.8 vs. 8.5%) and a longer duration of diabetes (10.6 vs. 6.4 years) and were treated more frequently with diet only and less frequently with insulin. Patients who did not give informed consent did not differ from those who gave informed consent.

Intervention (the quality improvement program)

The quality improvement program was implemented in 1993 and consisted of four elements.

First, the GPs received guidelines on the structure of diabetes care, targets for glycemic control and cardiovascular risk factors, and therapy according to a step-up regimen to achieve those targets. The guidelines were in accordance with those of the Dutch College of General Practitioners (12) and the European NIDDM Policy Group (13).

Second, a program of structured meetings in two regional peer-review groups (14) was implemented, including postgraduate education on type 2 diabetes, consultation of experts, and audit and feedback. During these meetings, GPs were educated by experts on the implementation of the guidelines in clinical practice. In addition, the GPs were given feedback regarding patient outcomes and outcomes of the individual practices on the care provided, and obstacles encountered in providing diabetes care and achieving the targets in accordance with the guidelines were discussed. At the start of the study, the intervention focused primarily on glycemic control, but in 1995 and thereafter, more attention was given to cardiovascular risk factors (15,16).

Third, GPs were given diabetes templates to achieve structured registration of the care provided.

Fourth, a central recall system for annual control visits of the patients was implemented. The data collected during this control visit were sent to the GPs and were available at the time of the clinical encounter of the GP with the patient.

The GPs in the RG continued to provide usual care. They were familiar with the national guidelines for the treatment of patients with type 2 diabetes because these had been published in a GP journal (12). The patients in the RG were also invited for an annual control visit, and the data collected during this visit were also sent to the GPs.

Data collection

Data on care provided.

Data on care received by patients were obtained from the medical records, dating from 12 months before the intervention to 42 months after. Three data abstractors were trained to review the medical records, and standardized computerized data-entry equipment was used. The interobserver and intraobserver reliability was good (Cohen’s κ ≥0.80 for all outcomes). The abstracted outcomes included the number of diabetes visits and the measurements of HbA1c, blood pressure (BP), total cholesterol (TC), HDL cholesterol, triglycerides, creatinine, urine albumin, and weight. A diabetes visit was defined as a visit during which glycemic control was assessed. Glycemic control measurement during illness to trace a possible disorder in the glycemic control was not recorded as a diabetes visit.

Patient characteristics and outcomes.

At baseline, the GPs provided information about their patients, with regard to demographical characteristics, duration of diabetes, and diabetes treatment.

Patient outcome data were collected during the annual control visits. After a baseline measurement in 1993, follow-up data were obtained after 18, 30, and 42 months. Height and weight were measured, with the patients wearing light clothes and no shoes. BMI was calculated as weight (kg) divided by height (m) squared. BP was assessed with a random-zero sphygmomanometer (Hawskley-Gelman, London, U.K.).

HbA1c, TC, HDL cholesterol, and triglycerides in fasting blood samples were assessed in regional laboratories until 1995. HbA1c was assessed by means of various methods. In 1995 and thereafter, all laboratory tests were performed in one laboratory and all previous measurements of HbA1c were standardized to those (normal range 4.3–6.1%).

Outcome measures

Outcomes on care provided.

The number of diabetes visits was determined per patient per year of follow-up. In accordance with the guidelines, appropriate assessment of glycemic control was operationalized based on two different criteria: 1) at least one HbA1c measurement per patient per year, and 2) at least four diabetes visits per patient per year.

Adequate assessment of BP, TC, HDL cholesterol, triglycerides, creatinine, and urine albumin was defined as one measurement per year (13), and adequate assessment of weight was defined as four measurements per year.

Patient outcomes.

It was assessed whether the targets for good or acceptable glycemic control were achieved in accordance with the guidelines (HbA1c <7.0% and ≤8.5%, respectively) (13). Additionally, it was assessed whether the targets for good or acceptable TC levels (<5.2 and <6.5 mmol/l, respectively), good or acceptable HDL cholesterol levels (>1.1 and ≥0.9 mmol/l, respectively), and good or acceptable triglyceride levels (<1.7 and <2.2 mmol/l, respectively) (13) were achieved. It was also assessed whether the targets for good or acceptable systolic BP (≤140 and ≤160 mmHg, respectively), diastolic BP (≤90 and ≤95 mmHg, respectively) (17), and BMI (<25 and ≤27 kg/m2, respectively) (13) were achieved.

Statistical analyses

Baseline characteristics of patients in the two study groups were compared using χ2 tests, unpaired Student’s t tests, or Mann-Whitney U tests.

To determine the effectiveness of the intervention over the total follow-up period, multilevel analysis was performed (18). Using this technique, calculated effect sizes can be adjusted for the clustering of patients within one general practice, which leads to dependency of observations of patients who receive care from the same GP. Repeated measurements are clustered under one patient, and patients were clustered within one general practice. Therefore, in the present study three levels were defined in the multilevel analysis: 1) repeated measures (i.e., time), 2) patient, and 3) GP.

The parameters of interest were as follows: 1) the regression coefficient pertaining to the dichotomous predictor variable, indicating the effect of the intervention during the total follow-up period, and 2) the interaction between the aforementioned dichotomous predictor variable and the baseline value of the particular outcome, indicating whether the effect of the intervention varied for patients with different baseline values.

A logistic model was used for the dichotomous outcomes and regression coefficients were transformed to odds ratios (ORs) using eβ.

The analyses were adjusted for sex, age, duration of diabetes, region, treatment, and baseline value and were also performed for subgroups of patients who already had achieved the target for good glycemic control at baseline. All multilevel analyses were performed with MlwiN (1998, version 1.02.0002) (18).

Patient characteristics and dropout

The characteristics of the GPs and the patients at baseline are shown in Table 1. The patients in the IG were older and had lower diastolic BP.

Medical records of 62 patients in the IG and 27 patients in the RG were not available for various reasons (relocation, change of practice, or death). These patients did not differ from the patients for whom medical records were available with regard to the baseline values of the different patient outcomes. A total of 312 patients (83%) in the IG and 77 patients (74%) in the RG were included in the analyses.

During follow-up, patients in the IG were more often lost to follow-up because of death (8.0 vs. 5.2%) and relocation (6.7 vs. 1.3%) than patients in the RG (P < 0.05). Patients in the RG dropped out more often because of referral to specialist care than patients in the IG (19.5 vs. 6.1%, P < 0.05). In the RG only, the mean baseline HbA1c and mean diastolic BP of patients lost to follow-up were both higher than in patients who completed follow-up (8.3 vs. 6.7% and 92 vs. 84 mmHg, respectively, P < 0.05).

Outcomes on care provided

Differences in diabetes care between the IG and the RG are shown in Table 2. Patients in the IG received diabetes care far more in accordance with the guidelines over the total follow-up period. The greatest improvements in the IG were achieved in the first 18 months of the study. The greatest difference in care received was found for the measurement of BP. In the IG during the total follow-up period, BP was 12.08 times more likely to be measured at least once per patient per year. The slightest difference was found for the measurement of albumin in urine. In the IG, urine albumin was 2.43 times more likely to be measured once per patient per year.

Compared with the results for the total patient population, no differences in effect of the intervention on care received was found for subgroups of patients with different baseline HbA1c values (data not shown).

Patient outcomes

Differences in patient outcomes between the IG and the RG over 42 months are shown in Table 3. Patients in the IG were as likely to achieve the targets for good and acceptable control over the total follow-up period as patients in the RG for almost all outcomes. Surprisingly, patients in the RG were 2.3 times more likely to achieve the target for good or acceptable diastolic BP (OR 0.44 [95% CI 0.20–0.97]). However, in supplementary analyses, in which the values of the last measurements were carried forward for patients referred to specialist care, this significant difference between groups in the effect of the intervention disappeared (0.47 [0.20–1.12]).

No difference in the effect of the intervention on patient outcomes was found for patients with different baseline values (data not shown).

Only meager empirical evidence is available regarding the generally assumed relationship between compliance of GPs with guidelines for diabetes care and patient outcomes (9). In the present study on the effectiveness of a quality improvement program focused on improving both the provision of diabetes care and patient outcomes, the intervention significantly improved only the diabetes care provided by GPs. This favorable effect of the intervention was not accompanied by any positive effect on patient outcomes. There are several possible explanations for the findings.

First, structured care might not affect the inexorable progress of diabetes in the long term. Most previous studies on the effectiveness of intervention strategies that focused on improving diabetes care assessed only the short-term effects on glycemic control (9). It is possible that the positive effects found in these studies faded after a longer follow-up period, as was the case in the present study.

A second explanation might be that patients were selected differently for the IG and the RG. In the IG, patients were more often lost to follow-up because of relocation and death. In the RG, patients were more often lost to follow-up because of referrals to specialist care. In the RG only, mean baseline values for HbA1c and diastolic BP were higher in patients lost to follow-up than in patients who completed follow-up. It is most likely that GPs in the IG more often accepted the challenge to continue to provide diabetes care to patients for whom targets for glycemic control were more difficult to achieve than GPs in the RG. In the present study, this difference in the selection of patients could have resulted in an underestimation of the effect of the intervention on patient outcomes. However, the effect did not change in supplementary analyses in which the values of the last measurements were carried forward for patients who were referred to specialist care during follow-up.

Third, the lack of effect on cardiovascular risk factors could be due to a premature evaluation. At the start of the study, the primary objective of the intervention was to improve glycemic control. However, from 1995, more attention was paid to the importance of diagnosing and treating cardiovascular risk factors. It is possible that the evaluation of cardiovascular risk factors was premature, because patients were not exposed to the intervention long enough for any changes to be detected.

Limitations of the study

For this study, a controlled before/after design was chosen, because the focus was to compare the effect of the multifaceted intervention with usual care. The GPs were not randomized to an IG or a RG, which might have resulted in an overestimation of the effects (19). Randomization would have implied that motivated GPs who were willing to implement the complex intervention could have been allocated to the RG, which would have made the GPs in both groups more comparable. However, it could also introduce a loss of contrast between study groups, because GPs in both the IG and RG would have been motivated to improve their diabetes care. In the present study, the percentage of patients who received care in accordance with the guidelines at baseline was lower in the RG, although the numbers were also quite low in the IG. This probably indicates a difference in motivation, skills, or interest between the GPs in both study groups. Therefore, from the present study, it can be concluded that even improving the diabetes care provided by motivated GPs did not improve patient outcomes.

Another reason for not randomizing the GPs was that randomization of a small number of GPs does not guarantee comparability of groups with regard to potentially confounding factors.

In the present study, we dealt with a large loss to follow-up, but this is not expected to affect the power of the study substantially, because we used longitudinal analysis. Irrespective of the number of observations per patient, every patient is included in the analyses, and longitudinal analysis deals appropriately with the varying numbers of observations (20).

Obtaining data on outcomes on the care provided from medical records can lead to underreporting of the care delivered (21). In the IG, GPs were encouraged to record the care they provided. It is possible that GPs in the RG also improved the care they provided but did not record this care. However, we consider that, in the interests of continuity of care, it is necessary to record the care provided. Failure to do so leaves the GP without information and patient status cannot be reviewed on schedule.

The results of this study could be “confounded by indication,” because patients with poor glycemic control at baseline are expected to be reviewed more often than patients with good glycemic control, and there is more room for improvement. However, no difference was found in any effect of the intervention on both care received and patient outcomes in subgroups with different baseline glycemic control.

The complex quality improvement program focused on improving knowledge, skills, and organization of the GPs’ practices was very intensive. In addition to structure of diabetes care, the guidelines included targets for glycemic control and control of cardiovascular risk factors, as well as a step-up therapy regimen to achieve these targets. In the structured meetings with experts, the GPs were educated on the targets and the step-up therapy regimen and were given feedback on the outcomes of the individual practices. Moreover, at the time of the clinical encounter of the GP with the patients, the data collected during the annual control visit were available. However, the patient outcomes did not improve. There seems to be a missing link in transferring the favorable effects on care provided to patient outcomes. It is possible that the GPs did not undertake the appropriate steps in adjusting the dosage of medication in the patients. Previous studies that implemented a prompt for the need to change therapy during an encounter or more detailed information on achieving good control for the individual patient showed improved patient outcomes. However, these studies only showed effectiveness in the short term (22,23).

In addition to regular review and intensive management of the patient by the diabetes care provider, active patient participation is also an important component (24). The patient must adhere to lifelong medication regimens and lifestyle modifications with regard to diet, smoking habits, physical exercise, and self care (25,26). It is possible that the absence of a patient-oriented aspect in the intervention strategy implemented in this study contributes significantly to the observed lack of effect on patient outcomes (9).

Future research should focus on studying the long-term effectiveness of strategies aimed at improving the care provided and the professional-patient relationship, with special focus on supporting GPs to undertake the appropriate steps in adjusting the dosage of medication in the patients and on supporting patients on self-management.

This study was funded by the Dutch Organization for Scientific Research (NWO), research program “Quality and Care,” (grant 940-20-096).

We thank Jos Twisk for his statistical advice and the general practitioners and practice assistants who participated in the ExtraMural Clinic of the Vrije Universiteit for their cooperation in this study and for the hospitality shown to the researchers during their visits.

1
Ho M, Marger M, Beart J, Yip I, Shekelle P: Is the quality of diabetes care better in a diabetes clinic or in a general medicine clinic?
Diabetes Care
20
:
472
–475,
1997
2
Goyder EC, McNally PG, Drucquer M, Spiers N, Botha JL: Shifting of care for diabetes from secondary to primary care, 1990–5: review of general practices.
BMJ
316
:
1505
–1506,
1998
3
Grimshaw J, Freemantle N, Wallace S, Russell I, Hurwitz B, Watt I, Long A, Shelton T: Developing and implementing clinical practice guidelines.
Qual Health Care
4
:
55
–64,
1995
4
Hetlevik I, Holmen J, Midthjell K: Treatment of diabetes mellitus: physicians’ adherence to clinical guidelines in Norway.
Scand J Prim Health Care
15
:
193
–197,
1997
5
Stolar MW: Clinical management of the NIDDM patient: impact of the American Diabetes Association Practice Guidelines, 1985–
1993
. Diabetes Care
18
:
701
–707,
1995
6
Oxman AD, Thomson MA, Davis DA, Haynes RB: No magic bullets: a systematic review of 102 trials of interventions to improve professional practice.
CMAJ
153
:
1423
–1431,
1995
7
Wensing M, van der Weijden T, Grol R: Implementing guidelines and innovations in general practice: which interventions are effective?
Br J Gen Pract
48
:
991
–997,
1998
8
Griffin S: Diabetes care in general practice: meta-analysis of randomised control trials.
BMJ
317
:
390
–396,
1998
9
Renders CM, Valk GD, Griffin SJ, Wagner EH, van Eijk JThM, Assendelft WJJ: The effectiveness of interventions directed at improvement of the management of diabetes mellitus in primary care, outpatient and community settings. In Cochrane Collaboration, Cochrane Library. Issue 1. Oxford, U.K., Update Software, 2001
10
Konings GPJM, Wijkel D, Rutten GEHM: The Dutch standard for diabetes mellitus type II, does it work? [In Dutch].
Huisarts Wet
38
:
10
–14,
1995
11
World Health Organization: Diabetes Mellitus: Report on a WHO StudyGroup. Geneva, World Health Org., 1985 (Tech. Rep. Ser., no. 727)
12
Cromme PVM, Mulder Dzn JD, Rutten GEHM, Zuidweg J, Thomas S: NHG standard on diabetes mellitus type 2 [In Dutch].
Huisarts Wet
32
:
15
–18,
1989
13
Alberti KG, Gries FA: Management of non-insulin-dependent diabetes mellitus in Europe: a consensus view.
Diabet Med
5
:
275
–281,
1988
14
Grol RPTM, Mesker PJR, Schellevis FG (Eds.): Peer Review in General Practice: Methods, Standards, Protocols. Nijmegen, Nijmegen University Department of General Practice, 1988
15
Yudkin JS, Blauth C, Drury P, Fuller J, Henley J, Lancaster T, Lankester J, Lean M, PentecostPress V, Rothman D: Prevention and management of cardiovascular disease in patients with diabetes mellitus: an evidence base.
Diabet Med
13
:
S101
–S1021,
1996
16
Koskinen P, Mänttäri M, Manninen V, Huttunen JK, Heinonen OP, Frick MH: Coronary heart disease incidence in NIDDM patients in the Helsinki Heart Study.
Diabetes Care
15
:
820
–825,
1992
17
Alberti KG, Gries FA, Jervell J, Krans HM: A desktop guide for the management of non-insulin-dependent diabetes mellitus (NIDDM): an update: European NIDDM Policy Group.
Diabet Med
11
:
899
–909,
1994
18
Goldstein H:
Multilevel Statistical Models
. 2nd ed. London, Edward Arnold, 1995
19
Kunz R, Oxman AD: The unpredictability paradox: review of empirical comparisons of randomized and non-randomized clinical trials.
BMJ
317
:
1185
–1190,
1998
20
Zeger SL, Liang KY: Longitudinal data analysis for discrete and continuous outcomes.
Biometrics
42
:
121
–130,
1986
21
Stange KC, Zyzanski SJ, Smith TF, Kelly R, Langa DM, Flocke SA, Jaen CR: How valid are medical records and patient questionnaires for physician profiling and health services research?: a comparison with direct observation of patient visits.
Med Care
36
:
851
–867,
1998
22
Benjamin EM, Schneider MS, Hinchey KT: Implementing practice guidelines for diabetes care using problem-based learning: a prospective controlled trial using firm systems.
Diabetes Care
22
:
1672
–1678,
1999
23
Cagliero E, Levina EV, Nathan DM: Immediate feedback of HbA1c levels improves glycemic control in type 1 and insulin-treated type 2 diabetic patients.
Diabetes Care
22
:
1785
–1789,
1999
24
van den Arend IJ, Stolk RP, Krans HM, Grobbee DE, Schrijvers AJ: Management of type 2 diabetes: a challenge for patient and physician.
Patient Educ Couns
40
:
187
–194,
2000
25
Rosenstock IM: Understanding and enhancing patient compliance with diabetic regimens.
Diabetes Care
8
:
610
–616,
1985
26
Bourn DM, Mann JI, McSkimming BJ, Waldron MA, Wishart JD: Impaired glucose tolerance and NIDDM: does a lifestyle intervention program have an effect?
Diabetes Care
17
:
1311
–1319,
1994

Address correspondence and reprint requests to Carry Renders, MSc, Institute for Research in Extramural Medicine, Vrije Universiteit, Van der Boechorststraat 7, 1081 BT Amsterdam, the Netherlands. E-mail: CM.Renders.emgo@med.vu.nl.

Received for publication 27 December 2000 and accepted in revised form 20 April 2000.

A table elsewhere in this issue shows conventional and Système International (SI) units and conversion factors for many substances.