OBJECTIVE—To assess the performance of health systems using diabetes as a tracer condition.

RESEARCH DESIGN AND METHODS—We generated a measure of “case-fatality” among young people with diabetes using the mortality-to-incidence ratio (M/I ratio) for 29 industrialized countries using published data on diabetes incidence and mortality. Standardized incidence rates for ages 0–14 years were extracted from the World Health Organization DiaMond study for the period 1990–1994; data on death from diabetes for ages 0–39 years were obtained from the World Health Organization mortality database and converted into age-standardized death rates for the period 1994–1998, using the European standard population.

RESULTS—The M/I ratio varied >10-fold. These relative differences appear similar to those observed in cohort studies of mortality among young people with type 1 diabetes in five countries. A sensitivity analysis showed that using plausible assumptions about potential overestimation of diabetes as a cause of death and underestimation of incidence rates in the U.S. yields an M/I ratio that would still be twice as high as in the U.K. or Canada.

CONCLUSIONS—The M/I ratio for diabetes provides a means of differentiating countries on quality of care for people with diabetes. It is solely an indicator of potential problems, a basis for stimulating more detailed assessments of whether such problems exist, and what can be done to address them.

Designing simple, practical, and understandable ways to assess health system performance remains a challenging aspiration. Recognizing the multifunctional complexity of a health system, existing frameworks use a range of indicators to capture the different aspects of health systems (1). However, many of these indicators have no obvious direct link to health outcomes, and the policy implications are often unclear (2). A complementary approach involves the use of tracer conditions (3), which is based on the premise that focusing on carefully selected health problems makes it possible to identify weaknesses in elements of the health system and to obtain more direct insight into its performance.

The rising burden of chronic diseases worldwide demands measures that will capture differences in the care provided to those affected, and we propose diabetes as a suitable tracer condition; it is well defined, fairly easy to diagnose (4), and common. The prevalence worldwide is estimated to be 2.8% (2000) and expected to increase to 4.4% by 2030 (5), with this figure already exceeded in the U.S. (6). While mainly type 2 diabetes is increasing, type 1 diabetes is also increasing swiftly, at ∼3% per year, especially in Central and Eastern Europe and among young children (7).

Health system performance affects diabetes outcomes in several ways. Effective treatment reduces the risk of disabling or fatal complications (810). This is most apparent for type 1 diabetes in developing countries where access to insulin is extremely limited (11). However, it is also seen where health systems have collapsed, as in the former Soviet Union (12). Its optimal management requires coordinated inputs from a wide range of health professionals, access to essential medicines and monitoring, and, ideally, a system that promotes patient empowerment. This has relevance beyond diabetes. A health service that is unable to integrate these elements for management of diabetes is unlikely to be able to meet the needs of people with disorders such as asthma, epilepsy, or hypertension. We describe a means of using data on diabetes to make a “preliminary diagnosis” of the effectiveness of diabetes care and which is likely to be generally available to health systems. This will enable the identification of countries where there is a need for more detailed investigation to determine the scale and nature of any problems.

While many potential measures denote adverse outcomes of diabetes, the only one widely and routinely available is mortality. Mortality comparisons must however take account of diabetes incidence, which varies considerably between countries (13). Building on earlier work (14), we generate a measure of “relative mortality” that combines mortality and incidence, producing a mortality-to-incidence ratio (M/I ratio). The M/I ratio is commonly used in cancer epidemiology as a crude indicator of cancer survival or “case fatality” and thus the overall quality of health care.

Underrecording of diabetes on death certificates is well documented (15), particularly among the elderly, although certification at younger ages, usually with type 1 diabetes, is more reliable (16). Thus, to maximize diagnostic specificity, comparisons should be restricted to deaths at young ages. However, such deaths from diabetes are few, so the choice of an upper age limit requires a trade off between including a greater number of deaths from type 2 diabetes against the need to minimize random fluctuation due to small numbers. After inspecting the crude data over several years to assess the extent of variation with different cutoff points, we chose 39 years as the upper age limit for our primary analyses.

Mortality and population data were extracted from the World Health Organization mortality database (17). Data include deaths by sex, 5-year age band, and cause according to the ICD. We examined data for 29 countries for which uniform incidence data were available (see below). We calculated age-standardized death rates from diabetes per 100,000 population for the age-group 0–39 years (both sexes combined), using the European standard population (18). Death from diabetes was classified according to ICD 9th (250) or 10th revision (E10–E14). Here, we use a 5-year average over the period 1994–1998, the latest period for which data were available for all 29 countries. Exceptions were Germany, Canada, Israel, and Poland, where data were only available until 1997 or 1996 (Poland). Here, we used a 4- and 3-year average, respectively.

Published age-standardized incidence rates among children aged 0–14 years (per 100,000 population) were extracted from the World Health Organization DiaMond study, which collected standardized incidence data on type 1 diabetes for the period 1990–1994 using population registries in 50 countries (13). For Germany, Italy, and Latvia, we used published data from the EURODIAB study (19), which appeared to be more nationally representative. Where two or more centers in a country reported independently, we calculated a simple national average without weighting for sample size or regional sample fraction (which was unknown). We computed the M/I ratio for 29 countries, using the age-standardized death rate from diabetes for ages 0–39 years (1994–1998) and the published age-standardized incidence rate for ages 0–14 years (1990–1994) as described above.

Sensitivity analysis

To assess the robustness of the M/I ratio, we focused on the U.S., which had the second-highest M/I ratio among Western industrialized countries (Table 1). We developed a series of scenarios that assessed possible biases that might falsely elevate the U.S. M/I ratio: 1) overassignment of diabetes as an underlying cause of death in the U.S. (by 10, 20, and 50%), 2) underestimation of the “true” national incidence rate by using the average of three regional incidence rates, replacing this average by the highest regional incidence or the upper bound of the 95% CI, and 3) a combination of 1) and 2).

Validation

To externally validate our findings, we also present specific survival data from cohorts of subjects with diabetes from countries included in our study, as published by the Diabetes Epidemiology Research International Study of mortality among young people (aged <40 years) with insulin-dependent diabetes, followed until 1990, for Israel, Finland, the U.S., and Japan (20) and from a cohort study from the U.K. of age-specific mortality rates among insulin-treated patients aged 30–39 years (21).

Figure 1 shows a 20-fold variation in the age-standardized incidence rates across the 29 countries included in our analysis, from a low of <2 cases annually per 100,000 population in Japan to >35 per 100,000 population in Finland. Substantial independent variation in mortality is also apparent, with the pattern very different from that of incidence, ranging between 0.14 deaths per 100,000 population per year in Greece (medium incidence) and levels >10 times that in Russia (very low incidence), Estonia (medium incidence), and Finland (highest incidence).

Converting these data to a M/I ratio (Fig. 2), we find a large variation, with highest ratios in former communist countries of Central and Eastern Europe, along with Japan and the U.S. Individual cohort data (Fig. 2, insert) demonstrates mortality among young people (aged <40 years) with insulin-dependent diabetes, followed until 1990, to be much lower in Israel and Finland than in the U.S. or Japan, at 158 and 250 per 100,000 person-years compared with 408 and 760/100,000 person-years, respectively. The ratios of the rates for those aged 30–39 years in these cohorts to that from the U.K. cohort (1.3, 1.7, 3.9, and 5.2) are fairly similar to those obtained by comparing their national M/I ratios (aged 0–39 years) to the U.K. M/I ratio (1.7, 1.7, 3.3, and 7.8). This may be interpreted as an additional external validation of our approach.

The sensitivity analyses (Table 1) indicate that even assuming a relative overassignment of diabetes as a cause of death among those aged 0–39 years in the U.S. of 20% and a high incidence rate of type 1 diabetes among 0- to 14-year-old children of 17.8/100,000 population (as reported for Allegheny County, PA), the M/I ratio would still be twice as high in the U.S. as in the U.K. or Canada.

We present a measure that aims to capture differences in health system performance using (routinely) available data. We find a remarkable variation across countries, suggesting gross differences in the ability of health systems to provide adequate care for people with diabetes. It seems very likely that those countries with high values of the M/I ratio (which may be defined to be >0.1) are not performing well in delivering effective care for diabetes and that as a group they are distinctly different from those with low values (which may be defined to be <0.025). The intermediate group spans a fair range, and their “true” relative performance, especially toward the lower end of the scale, is less obvious. However, our sensitivity analysis suggests that the difference between a country toward the upper end and the low group is probably not artifactual.

Of course, the composite nature of the M/I ratio, using data from (mostly) different individuals spread across different mortality cohorts, means that no firm conclusions can be drawn solely from these figures, a constraint exacerbated by the variability resulting from small numbers of deaths in some countries. Indeed, producing a ranking as such is not the purpose of a tracer condition. Rather, the relative position of a country is to be interpreted as a warning flag signaling the need for a more in-depth assessment and a tool for monitoring progress in the longer term.

There are obvious potential uncertainties about the index used in our analysis. Even using a high maximum age of 39 years, some countries accumulated <50 deaths from diabetes over 5 years; but very large numbers of deaths in some widely separated anchor points suggest that chance variation is not a prime explanation for this spread (e.g., 1,045 deaths in Russia, 588 in Japan, 7,628 in the U.S., 607 in the U.K., and 469 in Italy). We are not able to exclude differentials in diagnosis and coding across countries, although this problem is at least partially addressed by the sensitivity analyses. There is good evidence that for at least some of the countries at opposite ends of the M/I scale coding practices are rather similar (15), and other work (14) noted similar findings for data based on deaths at ages 0–24 years during the 1980s. In addition, comparisons with survival data from prospective cohorts are congruent with our population-level observations (Fig. 2), providing an important external validation of our approach.

Next steps: the system response to a “positive” trace

The identification of a problem using a health system’s M/I ratio is only the first step. The next step involves examining complementary data to understand the immediate causes of death driving the differences and links to possible underlying organizational and system weaknesses or failures. Thus, much of the excess mortality in Japan was found to be due to diabetic renal disease (22), largely caused by higher incidence of end-stage renal disease and reduced access to dialysis compared with the U.S. (23). Lower survival rates among those with type 1 diabetes in Estonia and Latvia compared with Finland are explained by a higher proportion of deaths due to acute complications of diabetes (24), reflecting, among others, the greater experience in Finland where type 1 diabetes is much more common than in the Baltic states.

The U.S. has come under particular scrutiny because of their relatively poor outcomes (which are captured by our index), and available evidence suggests that organizational and social factors may be more prominent than clinical ones (25,26). It is also important to note that disadvantaged population subgroups may contribute to higher mortality (27), and in the U.S. complications and mortality from diabetes are known to be worst among the population of African descent (28,29). Yet, while the relative contributions of etiology, health care, and non–health care determinants to diabetes outcome are still inadequately understood (30), health care can play a substantial part in alleviating the health impact of disadvantage due to socioeconomic status (27,31) or ethnicity (32).

We fully recognize the limitations of measures such as the M/I ratio and emphasize that they are only indicators that should prompt more detailed investigation as noted above. It is, however, important not to lose sight of the initial premise underlying this work. Diabetes is both important in its own right and a tracer disease that can provide insights into the ability of the health system to respond to chronic disorders. The key elements of an effective response to each disorder are similar, including continuity of care; integration of care across primary, secondary, and social care interfaces; and active patient involvement. This is supported by the observation that the U.S. performs considerably worse than other industrialized countries, as assessed by premature deaths from not only diabetes but also hypertension and obstructive airways disease (33). However, comparisons of the care provided for these disorders pose even greater problems than diabetes when using routinely available data.

E.N. is the holder of a fellowship on international benchmarking awarded by the Nuffield Trust. C.B. was supported in part by a Leverhulme Visiting Fellowship from the University of Bristol. M.M.’s work on health systems performance is funded through the Health Systems Development Knowledge Program of the U.K. Department for International Development. However, none of the sponsors can accept responsibility for the views expressed.

1.
Arah O, Klazinga N, Delnoij D, Ten Asbroek A, Custers T: Conceptual frameworks for health systems performance: a quest for effectiveness, quality, and improvement.
Int J Qual Health Care
15
:
377
–398,
2003
2.
Walshe K: International comparisons of the quality of health care: what do they tell us?
Qual Saf Health Care
12
:
4
–5,
2003
3.
Kessner DM, Kalk CE, Singer J: Assessing health quality: the case for tracers.
N Engl J Med
288
:
189
–194,
1973
4.
Expert Committee on the Diagnosis and Classification of Diabetes Mellitus: Follow-up report on the diagnosis of diabetes mellitus.
Diabetes Care
26
:
3160
–3167,
2003
5.
Wild S, Roglic G, Green A, Sicree R, King H: Global prevalence of diabetes.
Diabetes Care
27
:
1047
–1053,
2004
6.
Engelgau MM, Geiss LS, Saaddine JB, Boyle JP, Benjamin SM, Gregg EW, Tierney EF, Rios-Burrows N, Mokdad AH, Ford ES, Imperatore G, Narayan KM: The evolving diabetes burden in the United States.
Ann Intern Med
140
:
945
–950,
2004
7.
Green A, Patterson CC: Trends in the incidence of childhood-onset diabetes in Europe 1989–1998.
Diabetologia
44
:
B3
–B8,
2001
8.
Diabetes Control and Complications Trial Research Group: The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus.
N Engl J Med
329
:
977
–986,
1993
9.
United Kingdom Prospective Diabetes Study Group (UKPDS): Intensive blood-glucose control with sulfonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS33).
Lancet
352
:
837
–853,
1998
10.
Writing Team for the Diabetes Control and Complications Trial/Epidemiology of Diabetes Interventions and Complications Research Group: Effect of intensive therapy on the microvascular complications of type 1 diabetes mellitus.
JAMA
287
:
2563
–2569,
2002
11.
Yudkin JS, Beran D: Prognosis of diabetes in the developing world.
Lancet
362
:
1420
–1421,
2003
12.
Telishevska M, Chenet L, McKee M: Towards an understanding of the high death rate among young people with diabetes in Ukraine.
Diabet Med
18
:
3
–9,
2001
13.
Karvonen M, Viik-Kajander M, Moltchanova E, Libman I, LaPorte RE, Tuomilehto J: Incidence of childhood type 1 diabetes worldwide.
Diabetes Care
23
:
1516
–1526,
2000
14.
Matsushima M, LaPorte RE, Maruyama M, Shimizu K, Nishimura R, Tajima N: Geographic variation in mortality among individuals with youth-onset diabetes mellitus across the world.
Diabetologia
40
:
212
–216,
1997
15.
Jougla E, Papoz L, Balkau B, Maguin P, Hatton F: Death certificate coding practices related to diabetes in European countries: the “EURODIAB Subarea C” study.
Int J Epidemiol
21
:
343
–351,
1992
16.
Will JC, Vinicor F, Stevenson J: Recording of diabetes on death certificates: has it improved?
J Clin Epidemiol
54
:
239
–244,
2001
17.
World Health Organization:
WHO Mortality Database.
Geneva, World Health Org.,
2003
. Available from http://www3.who.int/whosis/menu.cfm?path=whosis,inds,mort&language=english. Accessed 21 May 2003
18.
Waterhouse JAH, Muir CS, Correa P, Powell J:
Cancer Incidence in Five Continents.
Lyon, France, International Agency for Research on Cancer,
1976
19.
EURODIAB ACE Study Group: Variation and trends in incidence of childhood diabetes in Europe.
Lancet
355
:
873
–876,
2000
20.
DERI Mortality Study Group: International analysis of insulin-dependent diabetes mellitus mortality: a preventable mortality perspective.
Am J Epidemiol
142
:
612
–618,
1995
21.
Laing SP, Swerdlow AJ, Slater SD, Botha JL, Burden AC, Waugh NR, Smith AWM, Hill RD, Bingley PJ, Patterson CC, Qiao Z, Keen H: The British Diabetic Association Cohort Study. I. All-cause mortality in patients with insulin-treated diabetes.
Diabet Med
16
:
459
–465,
1999
22.
Diabetes Epidemiology Research International Mortality Study Group: International evaluation of cause-specific mortality and IDDM.
Diabetes Care
14
:
55
–60,
1991
23.
Matsushima M, Tajima N, LaPorte RE, Orchard TJ, Tull ES, Gower IF, Kitagawa T: Markedly increased renal disease mortality and incidence of renal replacement therapy among IDDM patients in Japan in contrast to Allegheny County, Pennsylvania, USA.
Diabetologia
38
:
236
–243,
1995
24.
Podar T, Solntsev A, Reunanen A, Urbonaite B, Zalinkevicius R, Karvonen M, LaPorte RE, Tuomilehto J: Mortality in patients with childhood-onset type 1 diabetes in Finland, Estonia, and Lithuania.
Diabetes Care
23
:
290
–294,
2000
25.
Tabak AG, Tamas G, Zgibor J, Wilson R, Becker D, Kerenyi Z, Orchard TJ: Targets and reality: a comparison of health care indicators in the U.S. (Pittsburgh Epidemiology of Diabetes Complications Study) and Hungary (Diabetes Care Hungary).
Diabetes Care
23
:
1284
–1289,
2000
26.
Klarenbach SW, Jacobs P: International comparison of health resource utilization in subjects with diabetes: an analysis of Canadian and American national health surveys.
Diabetes Care
26
:
1116
–1122,
2003
27.
Brown AF, Ettner SL, Piette J, Weinberger M, Gregg EW, Shapiro MF, Karter AJ, Safford M, Waitzfelder B, Prata PA, Beckles GL: Socioeconomic position and health among persons with diabetes mellitus: a conceptual framework and review of the literature.
Epidemiol Rev
26
:
63
–77,
2004
28.
Lipton R, Good G, Mikhailov T, Freels S, Donoghue E: Ethnic differences in mortality from insulin-dependent diabetes mellitus among people less than 25 years of age.
Pediatrics
103
:
952
–956,
1999
29.
Resnick HE, Valsania P, Phillips CL: Diabetes mellitus and nontraumatic lower extremity amputation in black and white Americans: the National Health and Nutrition Examination Survey epidmiologic follow-up.
Arch Intern Med
159
:
2470
–2475,
1999
30.
Chaturvedi N: Commentary: socioeconomic status and diabetes outcomes; what might we expect and why don’t we find it?
Int J Epidemiol
33
:
871
–873,
2004
31.
Edwards R, Burns JA, McElduff P, Young RJ, New JP: Variations in process and outcomes of diabetes care by socio-economic status in Salford, UK.
Diabetologia
46
:
750
–759,
2003
32.
Leggetter S, Chaturvedi N, Fuller JH, Edmonds ME: Ethnicity and risk of diabetes-related lower extremity amputation.
Arch Intern Med
162
:
73
–78,
2002
33.
McKee M, Nolte E: Responding to the challenge of chronic disease: ideas from Europe.
Clin Med
4
:
336
–342,
2004

A table elsewhere in this issue shows conventional and Système International (SI) units and conversion factors for many substances.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.