Struggles With Clinical Translation of Immune Intervention Trials

Immune intervention trials in type 1 diabetes have shown very mixed, and often surprising, results. Despite suggestions of benefit from GAD-Alum vaccine in a phase 2 study (1), further studies, including two large phase 3 trials, showed no effect (2–4). Phase 3 trials with Anti-CD3 monoclonal antibodies failed to achieve their primary outcome (5–7), despite very promising results from multiple phase 2 trials (8–13). The conflicting Anti-CD3 data may be explained by unfortunate changes in design (5) or dose (6,7) in phase 3 (14). Many immunologic strategies have been tested in phase 2 trials, some with signs of benefit, such as rituximab (15) and abatacept (16); others without benefit, such as mycophenolate mofetil with or without daclizumab (17) or anti-interleukin 1 blockade with either canakinumab or anakinra (18); and others with ambiguous effects, such as thymoglobulin (19) or alefacept (20). In this issue, there are two articles describing results from the phase 3 Efficacy Study of DiaPep277 in Newly Diagnosed Type 1 Diabetes Patients (DIA-AID 1) trial evaluating the safety and efficacy of a 24 amino acid peptide derived from heat shock protein 60, called DiaPep277 (21,22). According to Raz et al. (21) and Pozzilli et al. (22), it appears as if the study demonstrated a beneficial effect, and if so, that would be a very exciting finding. Yet, as outlined below, there are several aspects of the study that make that conclusion less certain. This commentary examines the studies and explores some clinically relevant issues readers may want to consider when interpreting the data from the trial.

The principal measure of efficacy in immune intervention trials in type 1 diabetes is preservation of C-peptide as an index of β-cell function. Almost all of the trials mentioned above have used a mixed-meal tolerance test (MMTT) to assess C-peptide response. The DIA-AID 1 study used two methods of stimulation of C-peptide, the MMTT and a glucagon stimulation test (GST) (21). The sample size was calculated from results of phase 2 studies, which had used the GST to stimulate C-peptide. Nonetheless, the initial primary outcome measure for the DIA-AID 1 trial was MMTT-stimulated C-peptide. As MMTT was the original primary outcome measure, it was performed at randomization (month 0) and after 6, 12, 18, and 24 months—a total of five measurements. As GST originally was a secondary outcome measure, it was performed at month 1 (defined as “baseline” for the GST, but 1 month after the first treatment had been given) and at 12 and 24 months—a total of three measurements. The authors intended, initially, to have MMTT be the primary outcome measure. They performed the first MMTT before initiating treatment (a true baseline measurement), and conducted the test at more frequent intervals. However, the primary outcome measure appears to have been changed from the MMTT to the GST. Specifically, Raz et al. (21) stated that “the study protocol was amended, and the statistical analysis plan was planned and finalized before the study was unblinded, with the GST clearly defined as the primary end point.” The study did have two planned interim analyses to permit reestimation of sample size. However, according to the study’s history on ClinicalTrials.gov (NCT00615264), the primary outcome measure was changed after the last subject had completed the trial. One could certainly argue that there has to be a very good and compelling reason before consideration is given to change a primary outcome measure. In this case, and in fairness to the authors, they did provide a rationale. Specifically, results from a trial using DiaPep277 in a study of patients with latent autoimmune diabetes of adults (LADA) stimulated the change, as “it became apparent that there might be discrepancies between the two methods” (21). Regrettably, a full report of that study has not been published. Nonetheless, there are some results included in Pozzilli et al. (22) comparing GST and MMTT that may have supported the change. In the LADA study, the correlations between GST and MMTT at baseline and at 12 months were 0.9 and 0.85, respectively, which seems like strong correlations between the two methods. However, the correlation between GST and MMTT was only 0.48 for change in C-peptide between baseline and 12 months. Unfortunately, the LADA study was terminated early for futility, and only 46 subjects were available for analysis. A very relevant question is whether data from that incomplete study is sufficient to warrant the change in the primary outcome measure of a large phase 3 clinical trial.

Pozzilli et al. (22) compared the GST and MMTT, noting a number of differences between these two stimuli. They highlighted the fact that the C-peptide response during an MMTT varies depending on the fasting plasma glucose, and they wondered whether variations in gastric motility or incretin hormone response might influence the outcome of an MMTT. In point of fact, people consume meals and need β-cell function to respond to these—an MMTT evaluates that. But again, one could argue that no one consumes or injects glucagon routinely and that the GST response may not be the clinically most relevant parameter. Moreover, a multitude of recent clinical trials of immune interventions in recent-onset type 1 diabetes have been reported and almost all have used the MMTT as the primary outcome measure (1–4,6–9,12,13,15–20). Each reader will have to make his or her own decision on what is the best test, as data comparing the two stimuli are not available from other large trials.

The conflicting results from the two outcome measures used in the DIA-AID 1 trial creates a difficult conundrum, as depending on what parameter is truly the best one to use, we could now ask as to whether there was or was not a beneficial effect. The interpretation is even more complicated by additional issues. First, there were 457 subjects randomized. However, there were only 330 (72% of those randomized) included in the analysis of the primary efficacy end point (21) and only 297 (65% of those randomized) of these were available for the comparison of MMTT and GST (22). Second, the outcome measures needed to be imputed for a fair number of subjects due to missing data. Moreover, although it is stated that there was a clinical benefit of DiaPep277 in terms of hypoglycemia, it appears the only statistical support provided is for rate of change in hypoglycemic events from month 3 to study end. Yet, the absolute difference between groups in hypoglycemia event rates in the modified intention-to-treat cohort at 24 months is only 0.14 hypoglycemic event per month, or about 1.7 events per year. Since many of the hypoglycemic events included are the typical mild events that are readily treated, even if statistically significant, such a difference may not have clinical meaning.

Thus, as outlined, DIA-AID 1 (21,22) has strengths and weaknesses. One strength lies in the importance of the question regarding type 1 diabetes prevention and the difficulty in conducting such prevention studies for type 1 diabetes. Another strength is that there are no safety issues. A third strength is that this intervention is antigen based, and thus should not impair overall immune responses. One weakness is that data were available for only a relatively small proportion of the subjects enrolled. The major weakness is the concern on the end point used, as depending on whether one uses the GST or the MMTT, DIA-AID 1 either does or does not show a benefit of the intervention. Although the GST difference was positive statistically, it should be noted that the difference in C-peptide is relatively minor as there is a large drop in C-peptide in both the treated and control groups.

I see nothing wrong with having a negative primary outcome and discussing insights and results from secondary outcomes, mechanistic studies, and subgroup analyses. As a matter of fact, such insights provide important information for the design of further studies, even if the primary outcome is negative. In the case of DIA-AID 1, the results do indicate that the field could benefit from further comparisons of the MMTT and the GST within future trials of interventions in type 1 diabetes. The question at hand is whether that is in the context of a positive trial (based on GST) or a negative trial (based on MMTT).