Prognostic Accuracy of Immunologic and Metabolic Markers for Type 1 Diabetes in a High-Risk Population: Receiver operating characteristic analysis

The DPT-1 initially screened 103,391 relatives of patients with type 1 diabetes for ICAs. Of these, 97,273 were eligible because they did not have diabetes, provided an adequate blood sample, were in the required age range (3–45 years for first-degree relatives and 3–20 years for second-degree relatives), and had a qualifying relative. The staging of the study confirmed ICA positivity, measured IAA status, and assessed FPIR to IVGTT (10). If the subject was ICA-positive, had an FPIR less than threshold on two occasions, or had impaired fasting plasma glucose or IGT as determined by an OGTT, he or she was eligible for the parenteral insulin trial. A total of 339 eligible subjects were randomized and followed until diabetes onset or study end (5-year follow-up). All subjects (and/or their parent) signed a written consent form approved by the participating study center’s human subjects committee (3).

All assays were performed as previously described (3). Plasma glucose was measured by the glucose oxidize method. C-peptide levels were determined by radioimmunoassay. Measurable C-peptide was considered ≥0.2 ng/mL because this was the lowest level detectable by this method.

The peak C-peptide was the maximum point of all measurements. Area under the curve (AUC) C-peptide was calculated using the trapezoid rule. The definition of risk category by IVGTT was based on FPIR, using the sum of the plasma insulin values of the 1- and 3-min samples. FPIR was above threshold if ≥10th percentile for siblings, offspring, and second-degree relatives (≥100 μU/mL if age ≥8 years; ≥60 μU/mL if age <8 years) and ≥1st percentile for parents (≥60 μU/mL) (11). FPIR above threshold was required for eligibility. Homeostasis model assessment of insulin resistance (HOMA-IR) was calculated as the fasting insulin (mU/L) × fasting glucose (mmol/L)/22.5 from the mean of fasting insulin at 4 and 10 min and fasting glucose at 4 min before each IVGTT performed. The measurements of fasting glucose, fasting insulin, FPIR, and HOMA-IR come from the IVGTT sample. The presence of cytoplasmic ICA was determined by indirect immunofluorescence, and the subjects with titers of ≥10 JDF units were considered positive (12). IAAs were measured by competitive liquid-phase radioimmunoassay using polyethylene glycol precipitation. GAD65A and ICA512A levels were measured simultaneously by combined GADA and ICA512A radioimmunoassay. The cut points for positivity were set at indexes of 0.032 and 0.049, respectively.

Abnormal glucose tolerance (AGT) was determined by impaired fasting glucose or IGT at 2 h. Impaired fasting glucose is defined by a fasting glucose concentration ≥110 and <125 mg/dL. IGT is defined by an elevated 2-h plasma glucose concentration ≥140 and <200 mg/dL (3).

Diabetes was diagnosed according to the American Diabetes Association criteria (13).

The baseline characteristics of subjects were summarized and compared between progressors and nonprogressors by a two-sample t test or a Wilcoxon rank sum test. The global performance of each marker measured at baseline (as continuous variables) in predicting progression to type 1 diabetes within 5 years was summarized by the receiver operating characteristic (ROC) AUC, and results were presented as the mean and 95% CI. CIs of AUC that exclude 0.5 were considered to indicate significant results. Multiple logistic regression analysis to predict disease status was used to determine the weight to be given each marker in the model, which was then linearly added to give a value for the combined markers, on which the AUC was calculated. Areas under the ROC curves between the markers were compared by a nonparametric method (14). Pearson correlation was used to assess the correlation between the markers. All tests were two-sided. The P values <0.05 were considered to be statistically significant. Analyses were performed using SAS version 9.2 (SAS Institute, Cary, NC) software.

By design, subjects were confirmed ICA-positive with a low FPIR and/or IGT (n = 339), giving them a projected risk of >50% for progression to clinical diabetes over 5 years; the subjects were followed for a median of 3.7 years (interquartile range 2.15–4.76 years). A total of 139 subjects were diagnosed with type 1 diabetes in the follow-up period. The dropout rate was 6.3%. The subjects who were lost to follow-up before the end of the study were considered to be nonprogressors. Progressors demonstrated elevated autoimmune activity compared with nonprogressors at baseline (P < 0.01). Progressors also manifested statistically significant poorer metabolic function at baseline for most measurements, with the exceptions of fasting glucose (P = 0.91), fasting insulin (P = 0.95), and HOMA-IR (P = 0.91) (Table 1).

The performance of each marker in predicting progression to type 1 diabetes within 5 years given as estimated AUC, and its 95% CI is shown in Table 2. ICA titer, GAD65A titer, fasting insulin, HOMA-IR, and fasting glucose from both IVGTT and OGTT performed poorly and did not demonstrate prognostic ability with lower CI limits of AUC <0.50. FPIR demonstrated significant but moderate prognostic value with an AUC value of 0.61 (95% CI 0.55–0.67), as did FPIR/HOMA-IR with the same AUC value. Peak C-peptide and 2-h glucose derived from OGTT yielded the greatest AUC value of all examined metabolic indexes at 0.67 (0.61–0.72), followed by AUC C-peptide with an AUC value of 0.65 (0.59–0.71). IAA titer and ICA512A titer were the only two immunologic markers that provided significant prognostic value with AUC values of 0.66 (0.60–0.72) and 0.63 (0.56–0.69), respectively. In nonparametric comparison of ROC curve areas, no significant difference was found among IAA titer, ICA512A titer, FPIR, FPIR/HOMA-IR, peak C-peptide, AUC C-peptide, and 2-h glucose. As well, these markers had significantly higher predictive accuracy than ICA titer, GAD65A titer, fasting glucose, fasting insulin, and HOMA-IR (P < 0.001) individually.

We further evaluated the prognostic accuracy of the combined markers. The relationship among the markers was first evaluated with correlation analysis (Table 3). With the exception of the high correlation between peak C-peptide and AUC C-peptide (r = 0.96), the magnitude of the correlation among the markers was low or nonsignificant. Multivariate logistic regression modeling was then used to evaluate the overall prognostic performance of the combined markers (Table 2). The AUC for two significant antibody titer markers (IAA and ICA512A) was 0.69 (0.63–0.75); AUC was 0.62 (0.56–0.68) for two IVGTT markers (FPIR and FPIR/HOMA-IR). The predictive value for the three OGTT markers, including 2-h glucose, peak C-peptide, and AUC C-peptide, was 0.76 (0.70–0.81). Combining antibody titers and OGTT markers provided an AUC of 0.80 (0.73–0.87), which was significantly higher than any individual marker (P < 0.001). The AUC value derived from the combined markers of antibody titers and OGTT markers was also significantly better than the AUC from two antibody titer markers (P = 0.003) or two IVGTT markers or combining antibody titer markers (P < 0.001) and IVGTT markers (P < 0.001). However, it was not significantly better than the AUC derived from three OGTT markers, including 2-h glucose, peak C-peptide, and AUC C-peptide (P = 0.46). The addition of FPIR or FPIR/HOMA-IR in the model did not increase the prediction accuracy beyond antibody titer and OGTT markers, resulting in an AUC of 0.79 (0.71–0.84). Interestingly, adding age, sex, BMI, and relationship to proband (if sibling) with the biomarkers did not affect the prediction accuracy.

Analyses were also done for two subgroups: one with AGT and another with NGT but loss of FPIR. The AUCs with 95% CIs for individual markers are listed in Table 2. In the subgroup with AGT (n = 118), the OGTT markers (2-h glucose, peak C-peptide, and AUC C-peptide) provided similar levels of predictive accuracy as the IVGTT markers (FPIR and FPIR/HOMA-IR) and ICA512A titers. Interestingly, IAA had poor predictive performance in this subgroup. The AUC derived from the combination of antibody titers and OGTT markers was 0.76 (0.66–0.86); the AUC derived from the combination of antibody titers, OGTT, and IVGTT markers was 0.75 (0.66–0.84). The two AUCs were not significantly different (P = 0.89). In the subgroup with loss of FPIR (n = 221), both peak C-peptide and AUC C-peptide provided significantly higher predictive accuracy than FPIR or FPIR/HOMA-IR (P < 0.03). Peak C-peptide, AUC C-peptide, and IAA titers were better markers than 2-h glucose in predicting disease onset in this subgroup (P = 0.01, P = 0.01, and P = 0.03, respectively). The AUC derived from the combination of antibody titers and OGTT markers was 0.79 (0.73–0.86); the AUC derived from the combination of antibody titers, OGTT, and IVGTT markers was 0.80 (0.73–0.86). The two AUCs did not differ (P = 0.66).

This investigation into the natural history of type 1 diabetes, using the data from a large, contemporary prevention trial, characterizes the prognostic value of immunologic and metabolic markers in predicting progression to type 1 diabetes in a high-risk population who had demonstrated ICAs and experienced either loss of FPIR or AGT. These findings complement a previous research publication evaluating the metabolic markers in a low-risk population using DPT-1 oral trial subjects (9). The results of this secondary analysis of DPT-1 data reinforces the current paradigm of disease progression from abnormal metabolic activity to overt diabetes as a progressive deterioration of β-cell function, leading to a decline in insulin production and an elevation in glucose level (15–20). Although the previous studies have shown that an increase in insulin resistance was associated with the risk of type 1 diabetes, our findings demonstrate that HOMA-IR did not add to the prognostic value beyond FPIR in high-risk subjects (Table 2) (21–24). This may be due to high-risk subjects being in a later stage of preclinical disease development in which altered insulin secretion is a far more dominant feature relative to subtle insulin resistance.

The prognostic accuracy of OGTT glucose markers evaluated in this investigation was concordant with previous findings investigating the predictive value of metabolic markers for subjects without metabolic abnormality. In both populations with or without abnormal metabolism, 2-h glucose was found to have similar levels of accuracy as IVGTT FPIR/HOMA-IR in predicting type 1 diabetes. However, two other OGTT markers (peak C-peptide and AUC C-peptide) and one other IVGTT marker (FPIR) produced more robust results in high-risk subjects. This is likely due to the subjects being in a more advanced preclinical stage of disease progression such that the predictive value of insulin secretion had become more apparent (9). In contrast to the current analysis, the previous findings of Xu et al. (9) that examined a population positive for ICAs, but with normal metabolic function (possibly indicating an earlier preclinical stage of disease), peak C-peptide, and AUC C-peptide were not found to have predictive value. This suggests that the predictive value derived from these metabolic markers varies at different time points, depending on the stage of disease progression. Future investigations may be interested in examining a time-dependent ROC method for analysis. The lack of predictive value found for fasting glucose in the current analysis remained consistent with the findings in a low-risk population. Although fasting glucose may be preferred because it is convenient and low in cost, it did not provide significant prognostic accuracy in either population. This may be attributed to the fact that the majority of subjects in this population were diagnosed earlier in the course of the disease based on elevated 2-h glucose level but before elevated fasting glucose was detected.

When the immunologic markers were evaluated as quantitative variables, IAA titer and ICA512A titer demonstrated significantly better predictive performance than ICA titer or GADA titer in this population. However, the accuracy from individual or combined antibody titer markers was modest. Adding the antibody titer markers to the OGTT prediction model did not significantly improve the accuracy. Moreover, IAA titer was found not to provide significant prediction value in the subgroup that had AGT. These results indicate that autoimmune activity may not alter the risk of type 1 diabetes after the metabolic function has deteriorated.

In a previous study by Barker et al. (24), abnormalities in 2-h glucose from an OGTT were as sensitive and had better specificity in predicting diabetes compared with loss of FPIR. Our results from the subgroup analyses indicated that OGTT markers (peak C-peptide or AUC C-peptide) provided similar accuracy as FPIR levels in the subjects with AGT and had better prediction than FPIR in the subjects with loss of FPIR. As well, the results indicated that FPIR (as a continuous variable) did not contribute additional predictive value beyond OGTT in both the AGT group and NGT group. These studies contribute to a growing body of evidence that suggests metabolic markers derived from OGTT measurements are the most efficient and effective analytical method of determining the risk for progression to type 1 diabetes in those with known genetic and autoimmune risk factors. Future intervention trials may consider eliminating IVGTT measurements if an OGTT has been taken as an effective cost-reduction strategy. On the basis of these results and previous findings, the design of future trials identifying relatives with a higher risk of diabetes might be improved by evaluating a combination of OGTT metabolic markers in the population with ICA positivity and abnormal metabolism. Because of the increased predictive value from using multiple biomarkers, future research on risk grouping and identifying more promising treatment targets may be based on conditional multivariate methods, such as recursive partitioning analysis, to determine the optimal thresholds of prediction, which will allow for the division of subjects at risk into more homogenous groups. This study has several strengths and limitations. The subjects who were lost to follow-up before the end of the study were considered to be nonprogressors. This could be a possible source of bias as study subjects that left the study early could have subsequently developed disease. However, as the attrition rate was small, this bias was likely negligible.

OGTT markers assess the efficiency of the body to metabolize glucose and have been used as the gold standard for diagnosis of type 1 diabetes for decades. Recent evidence, however, reveals a degree of within-subject variability in the OGTT measurements (25). Thus, in a future study that relies on OGTT markers, within-subject variability can potentially affect sample size requirements, study procedures, and overall study costs. Consideration of this factor would need to be given when using the results of the current study to design a new trial.

ROC analysis provides the most comprehensive evaluation of diagnostic/prognostic accuracy available because it estimates and reports all of the combinations of sensitivity and specificity that a diagnostic/prognostic test is able to provide. Despite the secondary data analysis, the sample size is sufficient to provide adequate statistical power (≥80%) to detect a 10% of difference in the AUC ROC of the biomarkers. Additionally, the tests are considered robust because a nonparametric approach was used for the comparisons, which eliminates the need for applying assumptions on the underlying distribution.

Accurate prognostication is dependent upon sensitive and specific assays accompanied by an understanding of pathophysiology and the natural history of disease. In this investigation, we evaluated the prognostic accuracy of immunologic and metabolic indicators for subjects at high risk for progression to clinical onset of type 1 diabetes within 5 years. Results indicated that IAA titer, ICA512A titer, FPIR, FPIR/HOMA-IR, 2-h glucose, peak C-peptide, and AUC C-peptide had significant but modest prognostic value individually. The combination of three OGTT markers (2-h glucose, peak C-peptide, and AUC C-peptide) provided significantly better AUC values compared with any individual marker. Investigating the optimal cutoff points for the combined markers is warranted in order to provide guidance for clinicians and researchers in targeting more specific populations of individuals at high risk for progressing to type 1 diabetes in future prevention trials.

The DPT-1 was sponsored by the National Institute of Diabetes and Digestive and Kidney Diseases, the National Institute of Allergy and Infectious Diseases, the Eunice Kennedy Shriver National Institute of Child Health and Human Development, the National Center for Research Resources, the American Diabetes Association, and the Juvenile Diabetes Research Foundation.

No potential conflicts of interest relevant to this article were reported.

P.X. researched data, contributed to discussion, wrote the manuscript, and edited the manuscript. C.A.B., J.M.S., J.S.S., and J.P.K. contributed to discussion and reviewed and edited the manuscript. D.C. researched data and reviewed and edited the manuscript. J.P.K. is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.