Risk Scores for Type 2 Diabetes Can Be Applied in Some Populations but Not All

A total of 52 centers from 34 countries worldwide have contributed data on >190,000 individuals to the DETECT-2 database. Minimum requirements for participation were population-based surveys or large cohorts of employees, including information on at least 500 people, with all people without previously known diabetes having a 75-g oral glucose tolerance test. Undiagnosed diabetes was classified according to the 1999 World Health Organization criteria (20). The dataset includes information on the blood specimen used for the glucose measurement (venous whole blood, venous plasma, or capillary whole blood) and the method of glucose assay. The World Health Organization equivalence table was used to convert all results into plasma glucose equivalents (20). For all studies used in this analysis, the fasting time was comparable between 10 and 12 h.

For this analysis, nine datasets from the DETECT-2 database were selected, which were representative of people from a diverse range of ethnic backgrounds (Northern and Southern Europe, U.S., Indian subcontinent, Asia, Australia, Pacific Islands, and Africa) (2,8,21–27). For the Third National Health and Nutrition Examination Survey (NHANES III), only the subsample of individuals that underwent a 75-g oral glucose tolerance test was included (2). These samples are not necessarily representative of the whole population but are a convenience sample on which to test the risk score.

The risk score tested was the Rotterdam Predictive Model (RPM) (12). It was developed from the Rotterdam Study cohort (28) and was externally validated in the Hoorn study (29). It was chosen as the risk score for testing because it is simple, uses information collected during a routine consultation, and has been externally validated. The RPM includes information on age, weight, sex, and treatment with antihypertensive medications. The score was as follows:

• Age: per 5-year increment from 55 years: 2 points

• Male: 5 points

• Use of antihypertensive medications: 4 points

• BMI ≥30 kg/m²: 5 points

An individual with total score above 6 points was considered as high risk of having undiagnosed diabetes.

In each center, data collection was performed according to local ethical rules and according to the Helsinki Declaration.

All analyses were performed using SAS version 8.2 (SAS Institute, Cary, NC). The performance of the RPM was assessed when applied to each population. The particular performance characteristics examined included area under the receiver-operator characteristic (ROC) curve (AUC), sensitivity (proportion of people with undiagnosed diabetes having a positive screening test), specificity (proportion of people without undiagnosed diabetes having a negative screening test), positive predictive value (proportion of people tested positive having undiagnosed diabetes), and the proportion of the population identified as requiring further biochemical testing. Positive predictive value depends on the prevalence of diabetes and was calculated using the prevalence of newly diagnosed diabetes from each study. The ROC curve is sensitivity plotted against 1-specificity for each cutoff value (30). It provides a visual comparison of the test performance, and the AUC is a measure of diagnostic accuracy. The AUC represents the probability that within each study population, a randomly selected diseased individual has a higher value of the test than a randomly selected nondiseased individual. Approximate CIs for the performance outcomes were estimated by bootstrapping (1,000 bootstraps) (31). All analyses were applied to the 30- to 65-year age range.

The characteristics of the studies are shown in Table 1. The prevalence of undiagnosed diabetes ranged from 0.9% in Cameroon to 19.6% in the Western Pacific islands of Nauru and Tonga. The lowest mean BMI was observed in the Indian subcontinent (23.1 kg/m²) and the highest in the Western Pacific (34.9 kg/m²). The mean age ranged from 43.0 to 51.6 years.

Figure 1 and Table 2 show the performance characteristics of the RPM for each population compared with the results in the Dutch population (12). AUCs ranged from 0.53 to 0.70. The AUCs were similar in the Caucasian countries of Denmark, Spain, Australia, and the U.S., with a mean AUC of 0.70 (95% CI 0.68–0.72). This was not significantly different than the original result in the Dutch population (0.68 [0.64–0.72]). However, the RPM did not perform as well in discriminating between undiagnosed diabetes and nondiabetes in the non-Caucasian populations, with AUCs ranging from 0.53 in Africa to 0.62 in the Western Pacific population. The mean AUC for the non-Caucasian populations was 0.61 (0.59–0.62), which is significantly lower than for the Caucasian populations (P < 0.0001).

When the RPM-recommended cut point of >6 points was assessed, there was considerable variation in performance parameters (Table 2). Sensitivity ranged from 11.5 to 56.5%, specificity from 65.1 to 92.8%, positive predictive value from 1.7 to 25.4%, and the percentage of the population requiring further testing from 7.7 to 38.0%. Even the populations with satisfactory overall performance by the ROC curve showed considerable variation. Among the Caucasian populations, Denmark had the lowest sensitivity (41.9%) and the U.S. had the highest (56.5%) compared with 78% in the Dutch population. These differences were also reflected in the percentage of the population requiring further testing, which was also lowest for Denmark (17.1%) and highest in the U.S. (30.9%). Among the non-Caucasian populations, the Western Pacific had the highest sensitivity (51.4%) but also the lowest specificity (65.1%).

The factors contributing to the RPM score were examined to determine the reasons for the difference in performance. In all the Caucasian populations, the odds ratio of having undiagnosed diabetes increased by age, whereas in India and Africa, the odds ratios were higher among individuals aged 40–59 years compared with those aged ≥60 years (Table 3). The same tendency was observed in BMI. In India, there was a significantly increased risk of having diabetes if BMI was >22.5 kg/m² compared with a BMI <22.5 kg/m². In the Caucasian populations, BMI had very little impact when BMI was <25 kg/m² (Table 3). The use of antihypertensive treatment was positively associated in all populations with the risk of having undiagnosed diabetes and with odds ratios of the same magnitude.

Assessment of risk of undiagnosed type 2 diabetes is commonly used to identify individuals who should be recommended for further biochemical testing. Several risk assessment tools have been developed for this purpose using a combination of demographic, clinical, and sometimes biochemical information (12–17,32). These risk assessment tools have invariably been developed and tested in Caucasian populations.

This study has demonstrated that a risk assessment tool developed in a Caucasian population performs reasonably well in other Caucasian populations with similar distribution of risk factors, but not in other populations of diverse ethnic origin. The major reason for the lack of transferability of the risk score is differences of the impact of especially BMI and age on the prevalence of undiagnosed diabetes.

Even in the Caucasian populations, using the same risk score cut point gave substantial differences in sensitivity, specificity, positive predictive value, and percentage of the population requiring further testing. No attempt was made to modify the risk score to examine whether performance could be improved, since this was not the purpose of this study. However, because the ROC curves were relatively flat between a score of 5 and 6, it would be difficult to overcome the differences in performance parameters by adjusting the cut point alone. Because we used unweighted data from NHANES III, the predictive value and the percentage needing further testing are likely to be overestimated for the North American region. However, the overall performance measured as AUC, the sensitivity, and the specificity will not be affected. Decreasing the prevalence to 8% will decrease the positive predictive value from 18.7 to 15.1%, and the percentage that needed further testing will decrease from 30.9 to 30.4%. The prevalence of undiagnosed diabetes is low in Africa, which implies that the calculations are based on very few cases. This especially affects the sensitivity.

Few studies have assessed the performance of risk assessment tools developed in one country and then applied them to populations of different ethnic origins. Tabaei and Herman (32) developed a predictive equation in an Egyptian population and reported comparable performance when applied to a U.S. population. Their equation included a combination of demographic information and capillary blood glucose and had a sensitivity of ∼65% and specificity of 96% in both populations. The similarity in performance could be anticipated from the demographic similarities of the two populations, which had a similar age and mean BMI (29.8 kg/m² in Egypt vs. 28.4 kg/m² in the U.S.). Spijkerman et al. (33) assessed the performance of the Cambridge risk score in detecting undiagnosed hyperglycemia (fasting plasma glucose ≥7 mmol/l or HbA_1c ≥6.5%) in ethnic minority groups from the Indian subcontinent and the Caribbean subjects living in the U.K. In the original cohort, a Cambridge risk score cut point of 0.199 gave a sensitivity of 77% and specificity of 72%, with 30% of the population requiring further testing. Even after adjusting the cut point to 0.127 for the Indian subcontinent and to 0.236 in Caribbean subjects, the overall performance AUC, sensitivity, and specificity decreased.

This study has shown that a simple risk assessment tool developed in a Caucasian population does not perform well in populations of different ethnic origins with different clinical characteristics, emphasizing the need to develop ethnic-specific risk scores for screening for undiagnosed type 2 diabetes. A major aim of the international collaboration, DETECT-2, is to develop screening strategies applicable across ethnic regions throughout the world (19) that take into account ethnicity and differences in distributions of important risk factors for undiagnosed diabetes.

Australia: Paul Zimmet, Jonathan Shaw, International Diabetes Institute (AusDiab). Cameroon: Léopold Fezeu, Jean-Claude Mbanya. Denmark: Torben Jørgensen, Charlotte Glümer, Knut Borch-Johnsen, Research Centre for Prevention and Health (Inter99). India: A. Ramachandran, Diabetes Research Centre, M.V. Hospital for Diabetes, Chennai, India. Korea: Nam H. Cho, Department of Preventive Medicine, Ajou University School of Medicine (The Korean Health Study); Kyu C. Kimm, Korean National Genome Institute–Korean National Institute of Health. Nauru: Paul Zimmet, Jonathan Shaw, International Diabetes Institute. Spain: Conxa Castell, Servei d’Educacio Sanit. I Prog. de Salut., Advisory Committee on Diabetes in Catalonia, Department of Health. Tonga: Taniela Palu, Stephen Colagiuri, National Diabetes Center (Tonga). U.S.: Yiling Cheng, Mike Engelgau, Centers for Disease Control (NHANES III).

Stephen Colagiuri, Knut Borch-Johnsen, George Alberti, Rhys Williams, Gojka Roglic, Beverley Balkau, A. Ramachandran, and Jaakko Tuomilehto.

The secretariat comprise Knut Borch-Johnsen, Charlotte Glümer, and Dorte Vistisen, Steno Diabetes Center.

The DETECT-2 project (evaluation of screening and early detection strategies for type 2 diabetes and impaired glucose tolerance) was undertaken on the initiative of the World Health Organization and the International Diabetes Federation.

We wish to acknowledge and thank all the centers that provided data for the DETECT-2 database.