We read with interest the recent fixed-effects meta-analysis of the connection between rosiglitazone and both myocardial infarction and cardiac death and the lessons garnered from the Food and Drug Administration Advisory Meeting (1,2). We urge caution in data interpretation. Due to the diversity of the designs (eligibility, doses, duration of follow-up, concomitant medications), we view fixed-effects meta-analysis as inappropriate. Therefore, we reanalyzed 48 (not 42) eligible studies via a new random-effects method—a far more robust approach—and came to a strikingly different conclusion (3). A strong association with cardiac death was found, but there was no significant association with myocardial infarction. The use of a diagnostic test (such as Cochran Q) to decide between fixed versus random effects is invalid. A global type I error cannot be assessed by such a hybrid approach (4). The Cochran Q test lacks sensitivity and specificity when the number of studies is large and event rates low.
While meta-analyses are sound statistical tools for combining studies and purportedly protecting the public, they can cause spurious concerns. For any drug, different specialists may look at different side-effect profiles. Events are often examined serially with no consideration for error control. The risk-benefit ratio is rarely considered.
We strongly recommend the creation of independent safety and monitoring boards for all new pharmaceutical products to look at emerging safety and efficacy data throughout human studies, including postmarketing surveillance. Appropriate multivariate sequential methods should be used to control ongoing statistical error rates and to weigh both risks and benefits when serious safety issues emerge. Such boards should include biostatistical expertise (specifically, sequential clinical trials and meta-analysis), epidemiologists, patient advocates, medical ethicists, and appropriate clinical expertise. In this way, the interest of patients and manufacturers can be successfully balanced.
