Identifying Individuals at High Risk for Diabetes: The Atherosclerosis Risk in Communities study

In 1987–1989, the Atherosclerosis Risk in Communities (ARIC) study recruited a population-based cohort of 15,792 men and women 45–64 years of age from four U.S. communities (11). Human subjects research review committees approved the study, and all participants gave written consent. Follow-up visits, ∼3 years apart, were conducted in 1990–1992, 1993–1995, and 1996–1998.

We excluded 2,018 individuals who at baseline had diabetes on the basis of clinical diagnosis, diabetes medication use, or fasting glucose ≥7.0 mmol/l (an OGTT was not performed at baseline), 95 from underrepresented minority groups, 210 not fasting for at least 8 h, 1,696 with missing information on risk factors at baseline, 3,834 who had no follow-up or incomplete information at the end of the study to ascertain diabetes, and 24 who had temporally inconsistent reporting of a diagnosis of diabetes across visits, thus leaving 7,915 participants for the analyses.

We assessed diabetes and hypertension medication use, smoking, and parental history of diabetes (in either parent) by interview and obtained physical measures with participants fasting and with an empty bladder. BMI was calculated as weight/height² (kg/m²), and obesity was defined as a BMI ≥30 kg/m². Waist girth was measured at the umbilical level. Blood pressure was determined as the mean of two standardized measurements.

All analytes were determined at central laboratories according to standard protocols: plasma glucose by a hexokinase assay, insulin by radioimmunoassay (¹²⁵Insulin Kit; Cambridge Medical Diagnosis, Billerica, MA), and triglycerides and HDL cholesterol by enzymatic methods (12).

We defined incident diabetes by an OGTT (fasting glucose ≥7.0 mmol/l or a 2-h glucose ≥11.1 mmol/l) at the end of the follow-up (1996–1998) or as a report of clinical diagnosis or treatment for diabetes during the follow-up period (13,14).

In consonance with the National Cholesterol Education Program (NCEP) Adult Treatment Panel III definition of the metabolic syndrome (15), we defined central obesity as a waist circumference >88 cm (35 in) for women and >102 cm (40 in) for men; high triglycerides as ≥150 mg/dl (1.70 mmol/l); low HDL cholesterol as <40 mg/dl (1.03 mmol/l) for men and <50 mg/dl (1.29 mmol/l) for women; impaired fasting glucose as a fasting value from 6.1 to 6.9 mmol/l as well as, following recent American Diabetes Association recommendations (16), from 5.6 to 6.9 mmol/l; and raised blood pressure as ≥130/85 mmHg or use of medication for hypertension.

We produced risk functions for detecting incident diabetes on a randomly selected half of the sample (training sample) using logistic regression models. Risk factors considered were sex, ethnicity, parental history of diabetes, use of medication for hypertension, height, age, various measures of obesity (waist, weight, BMI, waist-to-hip ratio, each investigated one at a time), systolic blood pressure, fasting glucose, HDL cholesterol, triglycerides, and fasting insulin. Continuous variables were examined with their squared terms. Models were built by including, first, easily obtained clinical variables, leaving those requiring laboratory determination for a second phase. Starting with variables that predicted incident diabetes in univariate models, we constructed multivariable models in a forward manner, eventually including all variables whose addition produced an increment of at least 0.005 in the area under the receiver-operating characteristic (ROC) curve (AUC) (17).

Once best models were defined, we evaluated their diagnostic properties on the other random half of the sample (testing sample). To do so, we first estimated each subject’s probability of developing diabetes based on the derived risk functions. We next established rules to characterize differing degrees of risk based on cut points of these probabilities. These cut points were defined by fixing proportions (20, 30, 40, and 50%) of the testing sample that would be deemed screen positive. We then evaluated the risk functions and their derived rules in terms of AUC, fraction of total incident cases identified (sensitivity), specificity, and positive and negative predictive values.

We also examined similar diagnostic proprieties of rules based on the NCEP Adult Treatment Panel III metabolic syndrome definition and variations of it in the testing sample. We estimated 95% CIs for the AUCs, sensitivity, specificity, and predictive values using 500 bootstrap samples (18). All analyses were performed with SAS software (SAS Institute, Cary, NC).

At baseline, 56% of the 7,915 individuals studied were women, 85% were white, 27% were hypertensive, and 20% were current smokers. Median and interquartile range for various characteristics were as follows: age 54 years (49–59); BMI 26.6 kg/m² (23.7–30.1); waist, women, 93 cm (84–103); waist, men, 97 cm (91–104); height 168 cm (161–175); systolic blood pressure 120 mmHg (108–133); fasting glucose 5.44 mmol/l (5.11–5.83). Comparing these characteristics with those of the 5,764 subjects excluded, the only characteristics suggesting possible important systematic differences were a greater percentage of African Americans, smokers, and hypertensive subjects among those excluded. Only 2% of the final sample was taking cholesterol-lowering medication.

We ascertained 1,292 cases of incident diabetes: 189 (cumulative incidence of 23.6%) among African-American women, 93 (22.5%) among African-American men, 532 (14.6%) among white women, and 478 (15.7%) among white men. Of these cases, 387 (30%) were ascertained by self-report of clinical diagnosis or medication use for diabetes. Independent of this ascertainment, the OGTT identified 1,156 (89%) case subjects, 317 (24%) by fasting glucose alone, 511 (40%) by 2-h glucose alone, and 328 (25%) by both criteria. Of total case subjects, all except 15 who were ascertained by self-report of diabetes at ARIC interim visits were present at the last follow-up visit.

We initially defined two models: one including only clinically detectable elements not requiring laboratory evaluation and the other including only fasting glucose. Next, we defined two further models. The first combined elements of the two initial models and the second additionally contained HDL cholesterol and triglycerides. Neither BMI nor fasting insulin was included in these models because the additional contribution to the AUC, although statistically significant, was minimal for each. Models generating risk functions separately for African Americans and whites had generally similar β coefficients and are not reported.

The diagnostic properties of the risk functions were next evaluated in the testing sample. Figure 1A shows the percent of incident diabetes case subjects in each decile of estimated risk. Risk functions including laboratory measurements provided important risk stratification: 52% of case subjects were distributed in the two highest deciles of risk, and 15% were distributed in the five lowest deciles. Figure 1B illustrates the fraction of individuals in each decile of estimated risk who developed diabetes (positive predictive value) for each model. Participants classified in the 9th and 10th deciles of estimated risk by the model including lipids had, in fact, a 33 and 52% risk of developing diabetes, respectively; among individuals in intermediate-risk categories (deciles 6–8), risk ranged from 13 to 24%. The other risk functions performed slightly worse, more so for that composed only of clinical variables.

Table 1 presents diagnostic properties for each risk function, displaying rules based on cut points, chosen to permit the percentage of the population identified as at risk to vary from 20 to 50%.

Although differences are small, rules based on risk functions including laboratory measurements performed generally better, as reflected in the estimated AUCs for these models (Table 1). Predictive ability of the clinical variable only and fasting glucose only models was not significantly different (AUC 0.71 vs. 0.74, P = 0.2). Compared with the clinical variable only model, the model combining clinical elements with fasting glucose was more predictive (AUC 0.78, P < 0.001) and that including lipids was the best predictor (AUC 0.80, P < 0.001).

Table 2 shows the properties of rules based on the presence of elements of the metabolic syndrome. Rules attributing equal weights for each element of the metabolic syndrome produced somewhat less desirable diagnostic properties than rules based on the risk function including lipids (Table 1). For instance, the presence of the metabolic syndrome (three or more abnormalities) labeled 23% as positive and identified 50% of future cases of diabetes (sensitivity), whereas a rule derived from the risk function including lipids, labeling as high risk a slightly lower sample fraction (20%), correctly identified slightly more (52%) future cases. The rule with NCEP cut points also showed less overall predictive capacity than the risk function including lipids (AUC 0.75 vs. 0.80, P < 0.001).

Lowering the NCEP cut point for fasting glucose (≥5.6 mmol/l) in the definition of the metabolic syndrome did not improve overall predictability (AUC = 0.75), but produced rules labeling the greater fraction of the sample as at high risk, and as such, had higher sensitivity and lower specificity.

Also seen in Table 2 are the diagnostic characteristics of rules based on an alternative metabolic syndrome approach, derived from rounding of the β coefficients of an all-categorical variable model. We assigned 1 point for the presence of each element of the metabolic syndrome (except impaired fasting glucose) present, 1 additional point for obesity (BMI ≥30 kg/m²), and 2 points for a fasting glucose ≥5.6 mmol/l (or 5 points when ≥6.1 mmol/l). Rules based on this approach performed slightly better. For example, a score ≥5, labeling 22% of participants as at high risk, identified 54% of future cases of diabetes; and a score ≥3, labeling 46% of participants as high risk, identified 81% of future cases of diabetes. The AUC for this approach (0.78) was greater than that obtained for the original NCEP rule (0.75, P < 0.001).

When the best risk function, that including lipids, was evaluated in sex and ethnicity strata, AUCs were 0.79 (95% CI 0.76–0.82) for men, 0.81 (0.78–0.83) for women, 0.80 (0.78–0.82) for whites, and 0.76 (0.71–0.80) for African Americans. Additional analyses showed that developing a risk function containing lipids on a training sample using only African-American participants did not improve its performance in this ethnic group in the testing sample (data not shown).

Diagnostic properties were better when the analysis was based only on cases ascertained by clinical diagnosis or treatment. For example, the AUC for the risk function including lipids increased from 0.80 to 0.87; that based only on clinical variables increased from 0.71 to 0.78. Finally, we tested in whites a clinical score developed in the San Antonio Heart Study (8) composed of age, sex, family history of diabetes, BMI, HDL cholesterol, and hypertension and found an AUC of 0.80.

Clinical trials demonstrate that high-risk individuals, defined as having IGT, can reduce their risk of diabetes by more than half when offered a well-structured intensive lifestyle modification program (2,3). Diagnosing IGT requires an OGTT, a test of high specificity (92%) but low sensitivity (52%) in the prediction of diabetes (8).

Our results indicate that a rule defining high risk (9-year probability of developing diabetes ≥26%) based on the risk function composed of multiple variables including lipids had similar diagnostic properties (sensitivity 52% and specificity 86%, respectively), labeling 20% of the sample as high risk. These properties are generally consistent with those previously reported for similarly constructed rules (8,9). Slight differences between results of the three studies are probably accounted for by differences in population characteristics such as age and ethnicity, duration of follow-up, and diabetes definition.

Of note is that rules derived from other risk functions (Table 1) and from various clinical scores based on the metabolic syndrome definitions (Table 2) had similar, though somewhat poorer, diagnostic properties at cut points labeling a similar fraction (∼20%) as positive.

Although the best diagnostic properties found here were those derived from a risk function including lipid variables (AUC 0.80), the gain is small compared with those derived from a risk function without lipids (AUC 0.78) and, depending on the setting, may not justify the increased resources needed for the lipid measures. In settings in which HDL cholesterol and triglyceride measurements are readily available, rules based on the metabolic syndrome definition (Table 2) are valid, although slightly less predictive, alternatives to rules based on risk functions including these variables. Similar findings have been described recently (19). Yet, for those who prefer to classify risk based on the metabolic syndrome definition rather than by entering numbers into a clinical calculator or webpage, the losses are small: lower sensitivity (2%) and specificity (4%) and a slightly greater percent of sample deemed positive (3%).

The properties we found for the NCEP definition (sensitivity of 50% and specificity of 82%) are similar to those found for IGT (52 and 92%, respectively) and for the same NCEP rule (53 and 85%, respectively) in a cohort of white and Mexican-American men and women (20). The sensitivity of the NCEP definition found in Finnish men was lower (41%) and specificity was higher (90%), perhaps because the NCEP waist cut point was too high in that setting (21). Lowering the cut point of impaired fasting glucose to 5.6 mmol/l, as recently recommended by the American Diabetes Association (16), produced an equally predictive, in terms of AUC, but more sensitive NCEP-based rule. Slight manipulations of the NCEP definition improved its predictive power and might serve as alternatives to those clinicians who prefer not to use risk functions.

A risk function built strictly on clinical variables, although having less overall predictability (AUC 0.71), was able to derive rules with sensitivities ranging from 40 to 77% and corresponding specificities ranging from 84 to 55%. A reported score composed of a slightly different set of clinical variables also had good diagnostic properties (10), although, in that study, case definition was more stringent. Thus, rules based only on clinical information may be of value, for example, as a first step in serial diagnostic strategies for primary prevention in community settings.

Whether rules with ∼50% sensitivity, such as IGT and those mentioned above, detect an adequate number of future cases of diabetes for prevention is debatable. The main point against rules with greater sensitivity is the consequent increase in resources necessary for interventions. However, to optimize resource use, one could categorize more than just high- and low-risk groups and implement graded intensities of interventions, according to the degree of risk. Our data suggest that cut points for such categorization of risk in middle-aged U.S. populations might be between deciles 5 and 6, and 8 and 9. As illustrated in Fig. 1, participants with estimated risk in the 9th and 10th deciles had, in fact, a risk of developing diabetes over 9 years of ∼30 and 50%, respectively. In contrast, the risk of developing diabetes among individuals in the first five deciles ranged from ∼1% to ∼9%.

The large community-based sample of white and African-American men and women followed over the current epidemic phase of diabetes in the U.S. and the use of split samples to generate and validate rules presented strengthen the validity and generalizability of our findings. The nearly equivalent predictability of similar equations reported in other ethnic groups (8,9) suggests that these rules may also be applicable to other U.S. ethnic groups. In fact, the equation developed in non-Hispanic whites and Mexican Americans of the San Antonio Heart Study (8), when applied to our ARIC sample, produced an AUC (0.80) equal to that found when including a similar set of variables.

Yet, some limitations need to be considered. Losses to follow-up were not small. Although those lost presented a risk profile generally similar to those studied, their exclusion could possibly bias the diagnostic properties described. Because an OGTT was not done at baseline, some cases detected, especially early on, could be prevalent ones. Yet this, in fact, may increase the clinical relevance of our predictive equations, because undetected cases of diabetes not meeting the diagnostic criteria of fasting hyperglycemia are common in clinical practice. Additionally, our reported diagnostic properties would have been higher if only clinically diagnosed cases had been included. Finally, the sensitivities and specificities presented here for middle-aged adults, whites, and African Americans may not be applicable to younger or older groups or to those in other settings, resulting in a different underlying risk of developing diabetes.

In conclusion, rules derived from readily available clinical information, alone or combined with simple laboratory measures, can characterize groups of middle-aged adults as having various degrees of diabetes risk. This categorization permits grading the intensity of preventive actions according to the degree of risk of each patient. Though further validation of these rules in other samples is important, they have immediate application. In addition to their use in clinical encounters, they can be applied by managed care organizations to existing databases to identify high-risk individuals. Algorithms based on these rules can also facilitate enrollment in clinical trials testing new strategies to prevent diabetes.

Support for this study was provided by National Heart, Lung, and Blood Institute Contracts N01-HC-55015, N01-HC-55016, N01-HC-55018, N01-HC-55019, N01-HC-55020, N01-HC-55021, and N01-HC-55022 and National Institute of Diabetes and Digestive and Kidney Diseases Grant 5R01-DK56918-03. M.I.S. and B.B.D. received support from a Centers of Excellence Grant of CNPq (the Brazilian National Council for Scientific and Technological Development).

The authors thank the staff and participants in the ARIC study for their important contributions.