Understanding the Gap Between Efficacy in Randomized Controlled Trials and Effectiveness in Real-World Use of GLP-1 RA and DPP-4 Therapies in Patients With Type 2 Diabetes

Despite the development of nearly 40 new medications for type 2 diabetes during the past decade, only approximately half of adults with type 2 diabetes in the U.S. achieve a general target glycated hemoglobin (HbA_1c) of <7% (<53 mmol/mol), a proportion that has not improved during the last 10 years (1–3). The efficacy of these new medications has been demonstrated in many randomized controlled trials (RCTs) (4,5). However, real-world (RW) medications use may differ from use in RCTs, potentially contributing to poorer glycemic outcomes (6,7).

A 2004 systematic review of diabetes medication adherence indicated that 36–93% of patients were adherent (8), but more recent studies have found adherence rates to vary from 20 to 50%, depending on the assessment methods and drug class (9,10). Numerous studies have associated poor medication adherence and problematic long-term persistence (or discontinuation) with poor outcomes for patients with type 2 diabetes, including higher health care costs and utilization and poorer glycemic control (10,11). In contrast, patients enrolled in RCTs may be more likely to take their study medication as directed because of the support provided by the trial, and nonadherent patients are often removed from the published analysis. Patients enrolled in RCTs may also be more motivated to improve their health, leading to better medication adherence and other health-promoting behaviors, and may differ from the broader population of patients with diabetes in ways that influence outcomes (i.e., disease progression or baseline HbA_1c).

The objective of this study was to compare the change in HbA_1c between RW and RCT settings after initiating a glucagon-like peptide 1 receptor agonist (GLP-1 RA) or dipeptidyl peptidase 4 (DPP-4) therapy and determine the factors contributing to any differences observed. Both medication types have demonstrated a significant reduction in HbA_1c in RCTs, with additional benefits including lower incidence of hypoglycemia compared with sulfonylureas (12,13). Sodium–glucose cotransporter 2 inhibitors are another new drug class with similar benefits but were excluded owing to the limited number of patients taking sodium-glucose cotransporter 2 inhibitors included in the database used for this study.

To understand the factors associated with differences between RW and trial outcomes, this study compared patient characteristics (e.g., age) and behaviors (e.g., medication adherence). The concept of medication adherence, also called compliance, is intended to assess the degree to which patients follow their health care providers’ medication recommendations (e.g., prescribed dose and frequency) during a specified interval of time (14). The critical measure used in this study was based on the percentage of days covered (PDC), which is the most commonly used index in the adherence literature and is used in quality measures endorsed by the National Quality Forum and in Medicare Star ratings (15–17).

This study used simulated trial design framework (18) to compare patients enrolled in RCTs of DPP-4s or GLP-1 RAs with RW patients with type 2 diabetes and estimated a multivariate model of the RW change in HbA_1c levels. The estimated model, combined with the descriptive RW and RCT data, was used to describe differences between RCT and RW HbA_1c outcomes and the factors contributing to these differences (Supplementary Fig. 1).

The Optum/Humedica SmartFile database (spanning January 2007 to December 2014), one of the largest and most comprehensive integrated databases that includes administrative claims (pharmacy and medical) and electronic medical records, was used to analyze the change in HbA_1c (from the electronic medical records) among patients with type 2 diabetes initiating a GLP-1 RA or DPP-4 therapy in RW settings. The index date for each patient was the date of the first prescription fill for a DPP-4 or GLP-1 RA (index drug). The baseline period was the year before the index date, and the follow-up period was the period of ∼1 year between the index date and a subsequent HbA_1c measurement (± 90 days). Adult patients with type 2 diabetes (aged 18 years at the index date) were included if they had at least one prescription drug fill for a DPP-4 or GLP-1 RA therapy, continuous health plan enrollment (1 year before and after the index date), HbA_1c measured at the index date and ∼ 1 year later, and if they had key baseline characteristics similar to patients in enrolled in DPP-4 and GLP-1 RA clinical trials (baseline HbA_1c between 7 and 10% [53 and 86 mmol/mol] and at least one fill for a type 2 diabetes medication during the baseline period). Patients were excluded if they 1) had a diagnosis of type 1, secondary, or gestational diabetes mellitus during the baseline or follow-up periods; 2) had a fill for insulin during baseline; or 3) were diagnosed with dementia, hemiplegia, liver disease, metastatic solid tumor, AIDS, or a malignancy during baseline or follow-up (diagnosis codes in Supplementary Table 2). Patients initiating a GLP-1 RA or DPP-4 were selected based on National Drug Codes for each ingredient included in each class (GLP-1 RA: exenatide and liraglutide; DPP-4: sitagliptin, saxagliptin, linagliptin, and alogliptin).

The database was compliant with the Health Insurance Portability and Accountability Act. All data were deidentified and thus exempt from institutional board review.

A targeted literature search for published English-language RCTs was performed by searching pharmaceutical manufacturer websites for trials of GLP-1 RAs (liraglutide and exenatide) and DPP-4s (sitagliptin and saxagliptin) in patients with type 2 diabetes. Publications for other DPP-4 therapies (linagliptin and alogliptin) were excluded because <5% of RW DPP-4 patients in the study data set were treated with these drugs. Studies were limited to published manufacturer-sponsored trials because these trials form the basis for efficacy claims.

Inclusion criteria were applied to identify trials evaluating the efficacy of one of the above-listed drugs (often as an add-on therapy) in patients with type 2 diabetes initiating GLP-1 RAs or DPP-4s and previously treated with oral antidiabetic agents but not insulin. Trials in special patient subpopulations were excluded. A list of included and excluded RCTs, with the reason for exclusion, is in Supplementary Table 1. To ensure that the extracted clinical efficacy data reflected the effect of full/standard dosage of the medication, data collected from low-dosage treatment arms (0.6 mg) from two liraglutide trials (19,20) were excluded.

The data elements extracted from RCT publications included average age, baseline HbA_1c, sex, race, diabetes complications at baseline, type 2 diabetes therapy before GLP-1 RA or DPP-4 initiation, GLP-1 RA or DPP-4 drug and dose, follow-up duration, and change in HbA_1c. Publications were also reviewed to describe how the analysis handled patients who discontinued their index medication, did not comply with the drug regimen (poor adherence), or received rescue type 2 diabetes therapies during the follow-up period (e.g., due to high HbA_1c).

Change in HbA_1c levels was the primary study outcome, measured from a baseline reading near drug initiation (up to 90 days before or 14 days after drug initiation) to a second HbA_1c reading 365 ± 90 days after drug initiation (follow-up period). For patients with multiple HbA_1c readings near drug initiation, the observation measured closest to the index date was selected. Patients with multiple HbA_1c readings ∼1 year later (365 ± 90 days) contributed multiple estimates of change in HbA_1c but were appropriately weighted to ensure these patients did not contribute disproportionately to study findings.

Baseline characteristics included age (at the index date), baseline HbA_1c, use of advanced type 2 diabetes therapies (other than metformin monotherapy), and the presence of diabetes complications during the baseline year. The presence of diabetes complications was assessed based on the Diabetes Complications Severity Index (DCSI) score, a 13-point scale scored from diagnostic and laboratory data that is associated with diabetes progression and greater risk of death (21,22).

Adherence to the index drug was estimated based on the PDC with a nonoverlapping supply of the index drug (GLP-1 RA or DPP-4) during the follow-up period. Consistent with prior literature and standard quality metrics, patients were classified as adherent if the PDC was ≥80% (15–17). This study also measured discontinuation of the index drug, defined by the absence of the index drug on hand for at least 30 days and no subsequent fills through the end of the follow-up period.

Dosing of the index drug during the follow-up period was captured based on the last fill of the index drug in the follow-up period. Patients were classified as receiving at least a full or recommended dose based on each product’s prescribing information.

Other changes to the patient’s regimen of diabetes drugs (beyond the index drug) were also measured. Specifically, the addition and discontinuation of other (nonindex) type 2 diabetes drugs during the follow-up period was measured by comparing drugs on hand during the 90-day period before the first (index) and second HbA_1c measurements. The measure focused on drugs 90 days before each HbA_1c measurement because HbA_1c measures glycemic control during the previous 3 months; thus, drugs on hand during this interval were most relevant to the observed change in HbA_1c.

The multivariate model estimated the change in HbA_1c during the ∼1-year follow-up in RW patients, controlling for potentially confounding factors measured at baseline (HbA_1c, age, diabetes complications, and use of advanced diabetes therapies) and factors measured during follow-up (indicators of the addition of nonindex type 2 diabetes medications and adherence to the index drug). Sensitivity analyses were conducted to examine 1) the influence of race/ethnicity and sex, which were excluded from the main specification because these factors are correlated with adherence (23,24); and 2) the effect of excluding a measure of the use of advanced diabetes therapies at baseline. The model was estimated using ordinary least squares, and as a linear model, each coefficient demonstrates how much HbA_1c levels increased or decreased with every 1-point increase in the value of the covariate.

The estimated coefficients were used to calculate the predicted change in HbA_1c levels in trial and RW settings, estimate the gap, and describe factors contributing to the gap. The predicted change in HbA_1c in trial settings was calculated by multiplying each estimated coefficient by the value of that covariate in trials and summing the products. The predicted change in HbA_1c levels in RW settings can be estimated in the same way and is also mathematically identical to the actual RW change in HbA_1c levels. The gap between RW and trial outcomes is the difference between these two predicted values. The contribution of a variable to the gap is the regression coefficient multiplied by differences in the value of that variable observed in patients enrolled in trials and the RW. The predicted HbA_1c change in trial settings was compared with estimates extracted from the trials to assess the validity of the model’s predicted value. Three sensitivity analyses were conducted to assess how assumptions affect the gap between RW and trial HbA_1c outcomes and the role of medication adherence: 1) changes in patient selection criteria (included all patients with HbA_1c >10%); 2) assumptions about the percentage of patients in trials with any diabetes complications at baseline (varied from 5 to 30%), and 3) assumptions about medication adherence in trials (varied from 80 to 100%).

The analytic file and descriptive tables were created in SAS (version 7.1), and the regression analysis was conducted in Stata (version 14.2). Significance was assessed at P < 0.05. Categorical variables were compared with χ² tests and continuous variables with two-sided t tests. To account for multiple outcomes contributed by the same patients, all RW analyses (descriptive and regression analyses) were weighted such that each patient contributed equally to the results (weights were equal to the inverse of the number of HbA_1c measurements for a patient). SEs were corrected as appropriate (e.g., clustered SEs) to account for multiple observations of change in HbA_1c per patient.

The unit of analysis for the trial data was each relevant trial arm (e.g., for a GLP-1 RA or DPP-4), reported as a mean in the trial publication. These reported averages were then pooled, and the weighted average across all of the trials in each drug class was estimated. The reported SEs of the continuous measures (e.g., age, HbA_1c) were extracted from each arm, and an estimated grouped SE was calculated for each drug class. Trials that contributed multiple data points (e.g., two arms with different doses) received a weight summing to 1, where the weight was the inverse of the number of trial arms included in the mean. The maximum and minimum change in HbA_1c was reported for each drug class.

A total of 11 RCTs were selected (Supplementary Table 1), composed of 7 GLP-1 RA trials (5 of liraglutide [n = 2,243] and 2 of exenatide [n = 357]) (19,20,25–29) and 4 DPP-4 trials (2 of sitagliptin [n = 825] and 2 of saxagliptin [n = 845]) (30–33). A sitagliptin arm (n = 219) reported in a liraglutide trial (19) was also included. Only 1 of the 11 RCTs reported data on diabetes severity of patients enrolled in the study (25).

Data extracted from the selected RCTs are provided in Supplementary Table 3. All but one of the trials monitored patients for 24–26 weeks (Garber et al. [28] monitored patients for 52 weeks). All of the RCTs included protocols to rescue patients and excluded data from these patients by imputing data after rescue using last observation carried forward or by removing the patients entirely from the analysis, emphasizing the need to control for the addition of other medications in the RW.

Among 873 RW patients with type 2 diabetes meeting the inclusion criteria, 652 patients received DPP-4 therapy (contributing 911 observations) (Supplementary Fig. 2), and 221 patients received GLP-1 RA therapy (contributing 297 observations) (Supplementary Fig. 3). About half of GLP-1–treated patients initiated liraglutide (54%), and three-fourths of DPP-4–treated patients initiated sitagliptin. Most patients (80–96%) were receiving the recommended or maximum drug dose (Table 1).

Demographic characteristics were similar across data sources and treatments (Table 1). Baseline HbA_1c levels were similar among RW and RCT patients using GLP-1 RAs, but the mean baseline HbA_1c of patients in DPP-4 RCTs was lower than in RW patients. Baseline use of advanced therapy was more prevalent among RW patients compared with patients enrolled in RCTs (Table 1), suggesting that RCT patients may be earlier in the progression of type 2 diabetes than RW patients.

During the follow-up period, low percentages of RW patients treated with GLP-1 RAs and DPP-4s were adherent to their medications (29% and 37% were adherent, respectively), much lower than would be expected in trials (Table 1).

Approximately one-third of RW patients added another type 2 diabetes medication to their treatment regimens after initiation of GLP-1 RA or DPP-4 therapy (Table 1). These patients included possible rescues (among patients continuing their index drug) and possible switchers (among patients discontinuing their index drug). Medication discontinuation occurred in 40.3% (DPP-4) to 45.2% (GLP-1 RA) of RW patients (Table 1). Of the patients who discontinued GLP-1 RA or DPP-4 therapy, less than half (44% and 42%, respectively) added another drug, suggesting these patients switched to another medication. Of the RW patients who did not discontinue their GLP-1 or DPP-4 therapy, less than one-third (21% and 29%, respectively) added another drug, suggesting rescue therapy.

The overall unadjusted change in HbA_1c levels from baseline was similar among RW users of the two drug classes (GLP-1 RA: −0.52% [6 mmol/mol]; DPP-4: −0.51% [6 mmol/mol]) despite the greater efficacy of GLP-1 RAs demonstrated in RCTs (GLP-1 RA RCTs: −1.30% [−14 mmol/mol]; DPP-4 RCTs: −0.68% [−8 mmol/mol]) (Table 1).

Adherent RW GLP-1 RA patients demonstrated more than double the HbA_1c reduction compared with poorly adherent patients, with and without adjustment for potentially confounding factors (Table 1 and Fig. 1). The difference between adherent and poorly adherent patients was smaller for patients treated with a DPP-4.

Coefficients from the regression model used to estimate the change in HbA_1c level in RW patients are reported in Table 2 and show that baseline HbA_1c levels in both drug classes were important explanatory factors in the change in HbA_1c owing to their large (in absolute value terms) and statistically significant coefficients. The coefficient of −0.275 for baseline HbA_1c level in the GLP-1 RA regression indicates that for every 1-point increase in baseline HbA_1c levels, HbA_1c decreased by an additional 0.275% (2 mmol/mol).

Predicted change in HbA_1c using the values of covariates in the RCTs estimated a change of −1.03% (−12 mmol/mol) among patients receiving GLP-1 RA and −0.69% (−8 mmol/mol) among patients receiving DPP-4 (Fig. 2). These predicted values were within the range of the mean change in HbA_1c levels reported by the RCTs and 95% CIs of most of the selected RCTs. The gap between RW and trial change in HbA_1c levels was estimated to be 0.51% (6 mmol/mol; P < 0.01) for patients receiving GLP-1 RA therapy and 0.18% (2 mmol/mol; P = 0.07) for patients receiving DPP-4 therapy (Fig. 2). This was calculated as the predicted reduction in HbA_1c levels under typical trial conditions (GLP-1 RA: 1.03%; DPP-4: 0.69%) minus the RW change in HbA_1c (GLP-1 RA: 0.52%; DPP-4: 0.51%), yielding the gap of 0.51% for GLP-1 RA and 0.18% for DPP-4.

The degree to which each covariate contributed to the estimated gap is presented in Fig. 2. The efficacy gap can be explained by differences in adherence, the addition of another type 2 diabetes drug, and baseline characteristics (including HbA_1c levels, use of prior advanced therapy, age, and disease complications). Medication adherence accounted for ∼75% (0.39%) of the estimated 0.51% HbA_1c gap between RW and RCT patients receiving GLP-1 RA therapy and 72% of the gap between RW and RCT results among DPP-4–treated patients. Other factors may also play a role in explaining the gap between RW and trial outcomes, although any additional contribution to the gap is likely to be much smaller.

Sensitivity analyses around the inclusion of other factors in the model, including patients with high baseline HbA_1c (>10%) in the RW sample and varying assumptions about adherence and severity of diabetes among trial patients, did not alter conclusions regarding the efficacy gap between RW and RCT patients and the important role of medication adherence in explaining this gap. Previous RW studies have demonstrated sex and race/ethnicity differences in glycemic control among patients with type 2 diabetes, largely driven by differences in medication adherence (34–36). The inclusion of race and sex as variables in the current analysis slightly attenuated the effect of adherence on HbA_1c levels (sensitivity model 2 in Table 2). When patients with baseline HbA_1c >10% were included (estimates not shown), the estimated effect of adherence on change in HbA_1c from baseline became even more negative for the GLP-1 RA cohort but was similar for the DPP-4 cohort (the adherence coefficient changed from −0.581 to −0.706 in the sensitivity analysis in the GLP-1 RA regression and from −0.227 to −0.224 in the DPP-4 regression).

Adherence was assumed to be 95% in clinical trials in all of the results reported thus far. A final sensitivity analyses was conducted using the main model in Table 2 and varying the assumed adherence in trials from 80 to 100%. This sensitivity analysis found that adherence accounted for 72–75% of the gap, similar to the main findings.

Our findings demonstrate that the efficacy observed in GLP-1 RA and DPP-4 RCTs in patients with type 2 diabetes has not been fully realized in RW settings for either drug class, and poor medication adherence is a key reason. These results are consistent with a recent study of RW glycemic control in patients treated with liraglutide, which found an 0.80% reduction in HbA_1c among adherent patients and half that (0.42%) among poorly adherent patients (10)—similar to findings reported here for GLP-1–treated patients, more than half of whom were treated with liraglutide. Patients included in that study also had similar characteristics to this study: baseline HbA_1c was 8.22% (vs. 8.34% currently) and 34% of patients were adherent (vs. 29%) (10).

Four additional publications have examined RW change in HbA_1c among patients treated with liraglutide, exenatide, sitagliptin, and DPP-4s as a class (37–40); none of these examined the role of medication adherence, and the patient populations differed from our study and the previous study of medication adherence (10). Two of these studies selected patients that may bias toward patients more adherent than typical by requiring patients to have at least 3 months of fills and 6 months of follow-up (38,39). These two studies reported a change in HbA_1c of −1% to −0.9% for liraglutide, 0.68% for exenatide, and 0.63% for sitagliptin, similar to our findings among adherent RW patients, or a little lower in the case of exenatide (our study showed reduction of 0.86% for GLP-1s and 0.60% for DPP-4s). In addition, these studies reported HbA_1c reductions for liraglutide-treated patients and DPP-4–treated patients that were similar to adherent patients in our study. These two studies also monitored patients for less than 1 year (6–9 months) but did not appear to have required patients to have a minimum number of fills to be included (37,40). One study was limited to a single employer, and the other used matching techniques to select DPP-4–treated patients who were similar to patients treated with canagliflozin (this comparison was the focus of the study), potentially limiting the generalizability of the study to a broader patient population treated with a DPP-4 or liraglutide.

An additional study conducted in Europe, by Ahrén et al. (41), is worth noting because it explicitly compared trial HbA_1c outcomes with outcomes among patients treated with vildagliptin enrolled in a prospective “real-life” observational study. Ahrén et al. found that “real-life” patients treated with vildagliptin experienced similar reductions in HbA_1c as those enrolled in RCTs. However, these patients chose to enroll in a trial and may have had greater motivation to treat their diabetes and remain adherent to medication than typical RW patients. We, therefore, conclude that our finding that efficacy in RCTs is often not realized in RW settings, with poor medication adherence as a key explanatory factor, is consistent with prior literature.

This study identified medication adherence as a key explanatory factor for the gap between RW effectiveness and RCT efficacy. A drug cannot be expected to provide full benefit if the recommended therapeutic dose is not taken by the patient consistently. Reasons for poor adherence are likely multifactorial and have been previously noted to include issues such as adverse effects, cost, and patients’ medication beliefs (42). A large fraction of patients taking both oral (GLP-1) and injectable (DPP-4) agents were not adherent in this study. These routes of administration may be difficult to maintain for long periods in the treatment of chronic diseases such as type 2 diabetes. Innovation in dose delivery modality, particularly dose regimen simplification, has been previously associated with improved patient adherence via reduced dosing frequency, a lower rate of adverse effects, and increased safety (43–45).

The findings of this study should be considered in light of several limitations. First, patients were not randomly assigned to RW versus RCT settings; RW patients may differ in ways that may affect the ability to achieve lower glucose levels. Although we examined important parameters that can be measured and observed in the RW data, unobservable factors, such as disease duration and extent of progression, could bias results. RW patients appeared to be further progressed (more were previously treated with an advanced therapy compared with RCT patients), and these patients continued taking these therapies during follow-up. We controlled for the use of prior advanced therapy in the regression model, but it is possible that this metric did not capture the full affect of later use (and possibly poorer β-cell function), potentially understating the gap in glycemic benefit between RW and trial outcomes. RW patients may take more medications and have more comorbid conditions; these factors could contribute to the poor adherence observed in the RW, although the reasons for poor RW adherence were not examined in this study. There is evidence that older patients are more adherent to their diabetes medications (23). However, this study controlled for age, so this is unlikely to significantly bias results. Second, data on the use of medication samples were not available in the database. Third, medication adherence is likely overstated in claims data because administrative claims only measure filled prescriptions; not all medications dispensed can be confirmed to be taken by the patient. This is also likely to understate the role of adherence in explaining differences in outcomes in the RW and trial settings. Fourth, because this study did not control for differences between DPP-4– and GLP-1 RA–treated patients, comparisons across classes should be made with caution. The similar percentage of the gap explained by adherence for the two drug classes is likely an artifact of choosing the RW change in HbA_1c as the denominator of this percentage. Finally, although adherence to GLP-1 RA or DPP-4 was the study’s focus, many patients were taking other medications (e.g., metformin), and adherence to these drugs could affect results.

In conclusion, this study found that patients with type 2 diabetes receiving GLP-1 RA and DPP-4 therapies suffer from poorer outcomes in RW settings compared with their counterparts in RCTs, signifying an important gap in efficacy/effectiveness between trials and the RW. This result is unsurprising because numerous studies have documented differences between trial efficacy and RW effectiveness (46). The contribution of this report is to identify reasons for this difference. The multivariate model used in this study showed poor medication adherence was the most important reason for the lower effectiveness observed in the RW, which may also extend to other classes of type 2 diabetes medications. Glycemic control data have only recently become available in large RW databases. Future studies of RW outcomes in diabetes should measure and account for medication adherence, because this can be an important confounding factor. Oral and injectable administration routes both rely on consistent patient action to maintain adherence, yet studies such as this one frequently find low adherence rates. For chronic diseases such as type 2 diabetes, effective measures for improving adherence are often complex combinations of patient education, reminders, reinforcement, and convenience (47,48), which will necessitate sophisticated coordination of care (49). The results of this study indicate an urgent need for strategies to improve RW medication adherence so patients with type 2 diabetes can realize the full benefit of therapy and achieve optimal health outcomes.

Acknowledgments. Medical writing and copy editing assistance was provided by Shelley Batts, PhD, an employee of Analysis Group, Inc.

Duality of Interest. G.S.C., E.T., R.-D.T., and J.H. are employees of Analysis Group, Inc., which has received consultancy fees from Intarcia Therapeutics to conduct the study and for medical writing assistance. J.Y. is an employee of Intarcia Therapeutics and owns stock options. S.V.E. and W.H.P. are advisors to Intarcia Therapeutics and have not received any compensation for any work relating to this manuscript.

Funding for this research was provided by Intarcia Therapeutics, which was involved in all stages of the study research and manuscript preparation. No other potential conflicts of interest relevant to this article were reported.

Author Contributions. Manuscript drafts were prepared primarily by G.S.C. Data were collected by G.S.C., E.T., R.-D.T., and J.H. and were analyzed and interpreted by G.S.C., E.T., R.-D.T., J.H., J.Y., S.V.E., and W.H.P. All authors participated in the design of the study, had access to the data, and contributed to the manuscript development. All of the authors vouch for the accuracy and completeness of the data reported and the adherence of the study to the protocol, and all of the authors made the decision to submit the manuscript for publication. G.S.C. is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Prior Presentation. A synopsis of this study was presented as a poster at the 76th Scientific Sessions of the American Diabetes Association, New Orleans, LA, 10–14 June 2016.