We thank the authors of the two letters (1,2) for clarifications regarding the design of their observational studies reporting that use of sodium–glucose cotransporter 2 inhibitors (SGLT2i) in type 2 diabetes was associated with over 50% lower rates of all-cause mortality.
Thuresson et al. (1) suggest that immortal time bias was essentially absent in the study design of the Comparative Effectiveness of Cardiovascular Outcomes in New Users of SGLT-2 Inhibitors (CVD-REAL) study. I must disagree, particularly in view of some inconsistencies I found between key information provided in the letter (1) and that presented in the study report (3). In that article, the cohort is described as new users of an SGLT2i between November 2012 and November 2016 who were matched on the propensity score to new users of another glucose-lowering drug (oGLD) during that period (3). New users were defined by the first prescription for an SGLT2i or oGLD with no issued prescriptions of that medicine class during the preceding year. However, in the letter we are informed in the third paragraph that “we did not simply select the first oGLD prescription as the index date in the comparator group but instead selected a random initiation date” (1). These two conflicting definitions can both lead to immortal time bias, albeit to different extents.
Another unclear assertion in the letter is that “less than 50% of propensity-matched patients treated with SGLT2i had prior oGLD initiation during the study period (e.g., 46% in Sweden and 40% in Norway)” (1). Yet, in the article (3), the table displaying baseline characteristics reports that in the year before SGLT2i initiation, 79% were using metformin, 38% sulfonylureas, 33% dipeptidyl peptidase 4 inhibitors (DPP-4i), 9% thiazolidinediones, 20% glucagon-like peptide 1 receptor agonists, and 29% insulin, all drugs included in the oGLD group. It is difficult to reconcile these numbers with the statement that less than 50% of these patients had prior oGLD initiation and thus problematic to interpret the additional and sensitivity analyses presented in the letter.
Nyström et al. (2) suggest that time-lag bias could not be present in their study (4) because the Swedish treatment guidelines recommend earlier use of insulin. However, in view of often-important divergences between guideline recommendations and actual practice in many areas of medicine, such a pattern needs to be demonstrated using actual data to rule out time-lag bias. Moreover, simply including treatment duration and prior medications in the propensity score does not suffice to avoid such bias. There is not only an issue of confounding bias, which propensity scores address, but also selection bias, which propensity scores do not address. With respect to the question of incident versus prevalent users, the authors inform us in this letter (2), but not in their article (4), that insulin users were new users, with “a period of at least 1 year without the drugs of interest prior to the index date” (2). Finally, immortal time bias remains an issue when comparing SGLT2i with DPP-4i if patients using both drug classes were primarily included in the SGLT2i group and secondarily in the DPP-4i group. This same bias is also present in Nyström and colleagues’ new article on direct comparisons between SGLT2i and DPP-4i, as that study also uses a “hierarchical structure, starting with the dapagliflozin new user date” (5).
Several new CVD-REAL publications have replicated this hierarchical research design and reported, not surprisingly, practically identical results suggesting remarkable reductions in mortality with SGLT2i (5–7). However, these studies are also inherently affected by immortal time bias because they all used the same approach to study design and data analysis (8).
In essence, observational studies of drug effectiveness should mainly face the challenge of dealing with confounding bias due to the lack of randomization. Although confounding is generally inevitable in such studies, it can be minimized by the use of techniques such as propensity scores. On the other hand, immortal time and time-lag biases are preventable and should not be affecting observational studies; careful study design and data analysis will avoid these biases. Thus, whether SGLT2i drugs prevent one-half of all deaths in type 2 diabetes, as these real-world observational studies suggest, remains doubtful. Until observational studies are carefully conducted to avoid these biases, these spectacular findings, likely “too good to be true,” should be considered with great caution (9).
Duality of Interest. S.S. is the recipient of the James McGill Chair. S.S. has received research grants and has participated in advisory board meetings or as a speaker at conferences for AstraZeneca, Bayer Pharmaceuticals, Boehringer Ingelheim, Bristol-Myers Squibb, Merck, and Novartis.