The article by Yang et al. (1) published in a recent issue of Diabetes claims that diabetic patients on insulin treatment experience an extraordinarily small incidence of cancer compared with those not using insulin (rate ratio [RR] 0.18). The study was discussed in an editorial by Johnson and Gale (2), which was somewhat reserved toward the result for general plausibility reasons.
However, there is no reason for reservation just on general grounds; the study is fundamentally flawed and, by its very design, should be expected to give a substantial overestimate of the cancer risk among noninsulin users, hence the reported surprisingly small estimate of the RR for insulin versus noninsulin users.
The authors use a so-called new-user design as indicated in the reference (3). This reference merely proposes that the evaluation of drug effects should be carried out after exclusion of prevalent drug users to avoid confounding by duration (because duration is usually not known among prevalent users) and market age of drugs.
Yang et al. (1) do exclude prevalent users of insulin from their study, but unfortunately, also some of the follow-up time among nonusers, namely, the follow-up time among new users of insulin prior to insulin use.
This is best illustrated by first considering a follow-up of the entire register (restricted to individuals not on insulin at entry) as shown in Fig. 1. If we want to compare the occurrence rates of cancer between the nonuser and insulin groups, we must consider all follow-up time in the nonuser group when computing the rate of cancer. If we exclude follow-up time among those who later go on insulin therapy, we will overestimate the cancer rates among those not on insulin and, by that token, underestimate the insulin versus no insulin RR.
Illustration of the exclusion of risk time from the noninsulin group. Numbers in boxes refer to follow-up time (in person-years), and the numbers on the arrows refer to the number of transitions (events). Follow-up time was derived from Table 1 in Yang et al. as case subjects/rate. The correct follow-up time is in black, and gray numbers represent the erroneous allocation by Yang et al. in the full cohort analysis. DM, nonuser group; DM + ins, insulin group.
Illustration of the exclusion of risk time from the noninsulin group. Numbers in boxes refer to follow-up time (in person-years), and the numbers on the arrows refer to the number of transitions (events). Follow-up time was derived from Table 1 in Yang et al. as case subjects/rate. The correct follow-up time is in black, and gray numbers represent the erroneous allocation by Yang et al. in the full cohort analysis. DM, nonuser group; DM + ins, insulin group.
Full cohort analysis.
The authors, however, seem to make this error in their sensitivity analysis, deflating their follow-up time in the nonuser group and increasing it in the insulin group. The numbers in Fig. 1 suggest an inflation of noninsulin cancer rates by a factor 19,803/(19,803 − 2,380) = 1.14 and a deflation of the cancer rates in the insulin group of (3,137 + 2,380)/3,137 = 1.76. This gives an underestimate of the RR between users and nonusers of 1.14 × 1.76 = 2.00.
The reported RR from the total cohort is 0.48 (95% CI 0.32–0.73), so one would assume that a correct analysis of the entire cohort would give a result of about 0.96 (0.64–1.44) (because the relative uncertainty of the estimate is the same as the number of cancer events is unchanged).
The new-user design.
In their implementation of the new-user design, Yang et al. also exclude individuals who later go on insulin from the control group. Because the new-user group is by definition cancer free at insulin inception, the control subjects are selected from a group with an artificially high number of cancers.
In addition, “follow-up time of the nonusers was calculated as the difference between the follow-up time of the nonuser in the original cohort design and the time period from enrollment to the date of starting insulin of the corresponding user.” This means that further follow-up time in the nonuser group is excluded—follow-up time that by definition does not contain any events. Moreover, the follow-up time discarded is time prior to an arbitrary point in the follow-up for the nonusers, and it is purely defined by the matching insulin user.
Table 1 in the article by Yang et al. (1) shows that the matched group has 1,935/3,650 = 53% of the nonusers, 120/169 = 71% of the cancers, but a mere (120/0.0492)/(169/0.0097) = 14% of the follow-up time from the nonuser group. Hence, the study is by design expected to give a gross underestimate of the RR between users and nonusers of insulin, most likely substantially exceeding a factor 2.
Conclusion.
The bias in this study illustrates how errors arise from failure to recognize that the fundamental observational unit in follow-up studies is follow-up time and not individuals. Each small piece of follow-up (in principle, each day) should be classified by exposure status to insulin. Consequently, follow-up time from the same person may have different types of exposure.
In clinical trials, all follow-up time for a given person conveniently has the same exposure status, so the analysis becomes simpler. But analysis methods from clinical trials cannot be carried over to observational studies, which are essentially studies in the realm of medical demography. Demographic studies are best analyzed by demographic methods applied to all available follow-up data. Tailoring observational data to look like clinical trial data is at best inefficient and sometimes, as in this case, heavily biased.
ACKNOWLEDGMENTS
No potential conflicts of interest relevant to this article were reported.
