OBJECTIVE—To evaluate performance of process and outcome measures in the care of patients with diabetes seen in Department of Veterans Affairs (VA) facilities.
RESEARCH DESIGN AND METHODS—Retrospective audits of records and databases were conducted on randomly selected patients with diabetes over 5 years (1995 [baseline] and 1997–2000) in 22 VA networks. Performance on diabetes-specific and preventive processes was measured.
RESULTS—Nationally, significant improvements over time were observed for all measures (P < 0.001). For example, the percentage of patients receiving a dilated retinal examination rose from 44% in 1995 to 67% in 2000. The percentage of patients who received a urinary protein test rose from 23% in 1997 to 54% in 2000. Those who received influenza vaccination rose from 34% in 1995 to 78% in 2000. However, there was significant regional variation among all measures.
CONCLUSIONS—Adherence to diabetes-specific and preventive care measures in the VA improved from 1997 to 2000 compared with a 1995 baseline. The improvement occurred in a setting of the provision of guidelines, the contractual setting of specific targets, and the timely feedback of results to medical center and network directors. Future studies are needed to determine whether adherence to these measures will decrease the rates of complications in VA patients with diabetes.
The Department of Veterans Affairs (VA) is the nation’s largest integrated health care delivery system. It consists of >140 medical centers and >600 outpatient clinics administered through 22 regional networks and the VA Central Office (VACO) in Washington, D.C. (1). Diabetes is a major health issue in the VA. Approximately 17% of the 3 million veterans treated in the VA in 1998 had diabetes. Diabetes is also associated with significant morbidity and mortality in the VA. Nearly two-thirds of those patients who had a lower-limb amputation and 33% of those patients who received dialysis had diabetes (2).
Beginning in 1995, the VA initiated a major effort to transform the system from one oriented toward inpatient and specialty care to one that emphasizes outpatient and primary care (1). Key components of this initiative were: 1) assignment of patients to an identified primary care provider, 2) development of performance contracts for directors in the system, 3) development of clinical care guidelines for common conditions, including diabetes, and 4) implementation of electronic medical records, including computerized clinical reminders. In addition, to assess the effects of this initiative in patients with diabetes, the Office of Quality and Performance (OQP) in VACO designed a system known as the External Peer Review Program (EPRP), administered by an independent contractor, to assess and track diabetes-specific and general prevention measures throughout the VA. The diabetes-related results of this effort for the years 1995–2000 are reported here.
RESEARCH DESIGN AND METHODS
Performance measures
Diabetes-specific measures included the percentages of patients with an annual HbA1c; patient with an HbA1c <10% (1995–1998) or <9.5% and <8% (1999–2000); annual foot examination (inspection, palpation, and sensory evaluation); annual eye examination by an eye care specialist; and a urine protein test, including a microalbumin test if the initial urine test for protein was negative or trace. Other measures in these patients with diabetes included the percentage with an LDL cholesterol assay performed, an LDL cholesterol level <130 mg/dl among those in whom the test was performed, and a blood pressure <140/90 mmHg among those with an established diagnosis of essential hypertension. Further preventive measures included the percentages of patients with diabetes who were advised of the health risks of cigarette smoking, offered tobacco cessation counseling, immunized against influenza within the past year, and immunized against pneumococcal infection within the previous 5 years.
Improvement strategies
In addition to the measures above, several other mechanisms to improve the quality of care were initiated during the study interval. Evidence-based guidelines for diabetes were first (1996–1997) developed in collaboration with the executive committee of the National Diabetes Education Program, and subsequently with the Department of Defense. National VA headquarters instituted performance contracts with each network and medical center director that included fulfillment of reasonable goals to be achieved for each measure. Quarterly reports of the levels of the measures achieved were provided to the managers. Furthermore, most medical centers had the ability to use computerized clinical reminders, and all had the option to implement conferences, to select guideline champions, and to provide feedback to clinicians.
Determination of performance
Baseline data for each measure noted were obtained by VA’s Office of Quality and Performance by using ICD-9-CM codes to randomly select a sample of patients who received primary care visits (general medicine, primary care, and women’s health clinics) as outpatients in each of the 22 VA networks in FY1995 (1 October 1994 to 30 September 1995). To include only patients receiving continuous VA care, patients were selected only if they had had three or more visits during the previous year. Similar samples were drawn for comparison for the years 1997–2000. EPRP sample sizes varied between 8,513 and 25,764 patients per year. Patients sampled were further identified by age and sex. Because women constitute only a small fraction of the VA patient population (∼4%), they were oversampled in this selection process. Sample sizes were powered to permit statistical comparisons among the networks with the results provided to each of the 22 network directors. Data for each measure was obtained monthly by trained abstractors employed by an independent contractor (West Virginia Medical Institute, Inc., Charleston, WV) by chart review and from local databases. Data were tabulated quarterly and then sent to the appropriate managers.
Statistical analysis
To determine the statistical significance of linear trends for measures, the occurrence of a particular measure was considered as a dichotomous variable. The significance level for testing the null hypothesis of the absence of a linear trend was determined by calculating the correlation coefficient between the year and continuously measured dependent variables.
RESULTS
The mean age of men (65 years) and women (60 years) was comparable in each of the yearly samples. Results for the diabetes-specific measures are presented in Table 1. There was statistically significant improvement (P < 0.001) for the percentage of patients who had at least one HbA1c test annually, rising from 59% in 1995 to 94% in 2000. There were also improvements in all other diabetes process measures from the baseline period (1995) through 1998. For example, the percentage of patients with an HbA1c <10% improved from 72 to 87% from 1995 to 1998. From baseline to 1998, documented foot visual inspection rose from 77 to 95%, palpation of pedal pulses increased from 51 to 84%, sensory examinations increased from 38 to 78%, and rates of annual dilated retinal examinations rose from 44 to 67%. There was little change in the rates of adherence to these measures in the subsequent 2 years (1999 and 2000). There was also improvement in the measures that had a baseline collection period in FY1997. From 1997 to 2000, urinary microalbumin protein screening rose from 23 to 54%. LDL cholesterol measures rose from 47 to 89%, LDL cholesterol values <130 mg/dl increased from 62 to 76%, and the frequency of controlled blood pressure <140/90 mmHg in those with hypertension and diabetes improved from 40 to 52%.
Results for the preventive measures are presented in Table 2. There was statistically significant improvement between 1995 and 1998 (P < 0.001) in the percentage of all preventive measures. From baseline to 1998, immunization against influenza and pneumococcal infection rose from 34 to 70% and 29 to 68%, respectively. Smoking cessation counseling also increased from 39 to 90%, as did the frequency of nutritional counseling in obese patients (68 to 95%).
Evaluation of the 1998 data for each of the 22 geographic networks indicated significant regional variations in adherence to all diabetes-specific measures with the exception of the percentage of hypertensive patients with blood pressure <140/90 mmHg (Fig. 1).
CONCLUSIONS
Diabetes is a disease with serious outcomes that affects ∼17 million Americans (3) and has an annual cost estimated at nearly 100 billion dollars (4). Most of the morbidity and mortality of diabetes is due to the untoward outcomes associated with the disease: blindness, kidney failure, nerve damage, and cardiovascular disease. Recent controlled trials have shown the efficacy of a number of approaches aimed at decreasing the likelihood of these outcomes, including glycemic control (5,6), control of elevated blood pressure (7,8), decreasing raised levels of LDL cholesterol (9), screening for advanced retinopathy (10), and detection and preventive management of the high-risk foot to decrease the risk of amputation of the lower extremity (11).
As a result of the high burden of diabetes among patients cared for in the VA, this disease was targeted for quality improvement efforts in seven major categories: glycemic control, lipid control, blood pressure control, and screening for nephropathy, retinopathy, neuropathy, and vascular disease. Appropriate process and outcome indicators and a system to evaluate performance on these measures were developed. The evaluation process was aimed at increasing the frequency of screening and of adherence to process as well as intermediate health outcomes variables.
The results from this analysis show that there were clear improvements in all the diabetes process measures from the baseline period (1995) through 1998. However, there was little change in the rates of adherence to these measures in the subsequent 2 years (FY1999 and FY2000). The reasons for the plateau effect deserve additional study.
With the exception of retinopathy screening, it is difficult to benchmark these data against those from other health plans or population-based surveys from the mid- to late-1990s, although VA results are comparable to data through 1997 reported by the Indian Health Service (12). However, the VA performance measures for diabetes that were used during the period of this report had the same categories and thresholds as those subsequently developed by Diabetes Quality Improvement Project (DQIP) (13).
DQIP began under the sponsorship of a coalition of public and private entities (American Diabetes Association, Foundation for Accountability, Health Care Financing Administration, National Committee for Quality Assurance), and was later joined by the American Academy of Family Physicians, American College of Physicians, Centers for Disease Control and Prevention, and the VA. These measures have been incorporated into HEDIS 3.0 (14) and since 2000 have been used to accredit health plans.
In 2000, the first year the National Committee for Quality Assurance reported Comprehensive Diabetes Care Measures, the industry-best reported average was 43% for “poor A1c,” 66% for dilated retinal examination, 46% for LDL cholesterol controlled <130 mg/day, and 46% for renal screening (15). A recent study indicated that the percentage of individuals screened annually for HbA1c and semiannually for lipids was >20% higher for veterans 65–75 years of age than among comparably aged fee-for-service Medicare enrollees; however, retinopathy screening was 7% less (16). This would suggest that the VA would be comparable, if not better, than the best private-sector health plans.
For this reason, the efforts and results of VA in improving the quality of diabetes care provided to its patients should be relevant to other health care systems. This is particularly so for those systems that serve disadvantaged populations since the average patient in the VA is older, is of lower socioeconomic status, and has lower functional health status than the general population of the U.S. (17,18). Unlike the private sector, the VA has publicly reported data for both the DQIP accountability measures and those recommended for quality improvement, compared with the truncated measurement set utilized by HEDIS. In addition, the VA reports adherence to general health measures, such as smoking and immunizations, for the veteran population with diabetes.
Several strategies were instituted throughout VA to assist in achieving better performance on these measures, including computerized reminders and identifying a regular primary heath care provider for diabetes care. This is consistent with strategies that have been shown to improve diabetes-related process outcomes (19,20). Although it is not possible in VA to isolate the effect of each strategy, the simultaneous institution of multiple organizational changes appears to have contributed to the improvements seen throughout the system. These findings are consistent with the conceptual framework of policy driving systems-level changes in health care delivery (21).
While the frequency for all measures increased from baseline to 1998, significant opportunities for improvement remain. For example, the failure to decrease blood pressure to <140/90 mmHg for almost half of those patients with hypertension and the omission of retinopathy screening for more than a third of the patients indicate the need for continued efforts to improve adherence and for further research into better strategies to deliver effective health care (22).
In conclusion, our results suggest that a policy that mandates the effective use of guidelines, the measurement of readily available processes, and performance measures with appropriate feedback improves outcomes over the short term in a structured health care system. Longer-term studies are needed to demonstrate that greater adherence to these measures improves the outcomes of microvascular and macrovascular events in these patients.
Variability among VA networks in diabetes-specific process measures. Variability was tested using a standard χ2 test. Further analysis of variability in the achievement of the measures was assessed by displaying them in the form of a box plot; outliers for each distribution were identified (+) using as a criterion 1.5 times the midspread distance measured from the ends of the box.
Variability among VA networks in diabetes-specific process measures. Variability was tested using a standard χ2 test. Further analysis of variability in the achievement of the measures was assessed by displaying them in the form of a box plot; outliers for each distribution were identified (+) using as a criterion 1.5 times the midspread distance measured from the ends of the box.
Diabetes-specific measures expressed as percentages of patients meeting the measures for the baseline period (1995) and 1997–2000
Measure . | 1995 (n = 9,578) . | 1997 (n = 13,557) . | 1998 (n = 8,513) . | 1999 (n = 18,882) . | 2000 (n = 25,764) . |
---|---|---|---|---|---|
HbA1c performed | 59 | 85 | 91 | 93 | 94 |
HbA1c <10% | 72 | 82 | 87 | — | — |
HbA1c <9.5% | — | — | — | 82 | 85 |
HbA1c <8% | — | — | — | 59 | 62 |
Foot visual examination | 77 | 90 | 95 | 96 | 93 |
Foot pulses checked | 51 | 74 | 84 | 84 | 84 |
Foot sensation checked | 38 | 69 | 78 | 78 | 78 |
Dilated retinal examination by eye specialist | 44 | 55 | 62 | 67 | 67 |
Urine protein evaluated | — | 23 | 36 | 44 | 54 |
LDL cholesterol measured | — | 47 | 64 | 73 | 89 |
LDL cholesterol <130 mg/dl | — | 62 | 68 | 72 | 76 |
Blood pressure <140/90 mmHg if hypertensive | — | 40 | 44 | 45 | 52 |
Measure . | 1995 (n = 9,578) . | 1997 (n = 13,557) . | 1998 (n = 8,513) . | 1999 (n = 18,882) . | 2000 (n = 25,764) . |
---|---|---|---|---|---|
HbA1c performed | 59 | 85 | 91 | 93 | 94 |
HbA1c <10% | 72 | 82 | 87 | — | — |
HbA1c <9.5% | — | — | — | 82 | 85 |
HbA1c <8% | — | — | — | 59 | 62 |
Foot visual examination | 77 | 90 | 95 | 96 | 93 |
Foot pulses checked | 51 | 74 | 84 | 84 | 84 |
Foot sensation checked | 38 | 69 | 78 | 78 | 78 |
Dilated retinal examination by eye specialist | 44 | 55 | 62 | 67 | 67 |
Urine protein evaluated | — | 23 | 36 | 44 | 54 |
LDL cholesterol measured | — | 47 | 64 | 73 | 89 |
LDL cholesterol <130 mg/dl | — | 62 | 68 | 72 | 76 |
Blood pressure <140/90 mmHg if hypertensive | — | 40 | 44 | 45 | 52 |
Data are percent.
General preventive measures expressed as percentages of patients meeting the measures for the baseline period (1995) and 1997–2000
Measure . | 1995 (n = 9,578) . | 1997 (n = 13,557) . | 1998 (n = 8,513) . | 1999 (n = 18,882) . | 2000 (n = 25,764) . |
---|---|---|---|---|---|
Influenza immunization | 34 | 58 | 70 | 66 | 78 |
Pneumococcal vaccine | 29 | 56 | 68 | 68 | 81 |
Smoking cessation intervention | 39 | 78 | 90 | 85 | 59 |
Nutritional counseling if BMI >27 kg/m2 | 68 | 92 | 95 | 96 | 96 |
Measure . | 1995 (n = 9,578) . | 1997 (n = 13,557) . | 1998 (n = 8,513) . | 1999 (n = 18,882) . | 2000 (n = 25,764) . |
---|---|---|---|---|---|
Influenza immunization | 34 | 58 | 70 | 66 | 78 |
Pneumococcal vaccine | 29 | 56 | 68 | 68 | 81 |
Smoking cessation intervention | 39 | 78 | 90 | 85 | 59 |
Nutritional counseling if BMI >27 kg/m2 | 68 | 92 | 95 | 96 | 96 |
Data are percent.
References
Funding for this supplement was provided by The Seattle Epidemiologic Research and Information Center and the VA Cooperative Studies Program.
A table elsewhere in this issue shows conventional and Système International (SI) units and conversion factors for many substances.