In their recent report, Wells et al. (1) describe the derivation of a statistical tool designed to predict mortality risk in patients with type 2 diabetes. We have a number of concerns about the methods used and the reporting of this study. These concerns include the possibility that the extent of imputation used may have introduced bias, that the modeling is based only on baseline information, and that the report does not adequately reference prior work.
The researchers acknowledge that over 50% of some baseline characteristics of these patients were unavailable. The missing data have been imputed from the remaining available baseline data. In addition, the patients included in the dataset were not diagnosed with uniform criteria, the report did not include patients treated with diet alone, and only 2,000 of the 33,000 patients were followed for more than 5 years. Parameters for modeling long-term mortality risk from some variables may be derived from fewer than 1,000 patients.
A clinical mortality prediction model based only on baseline data may be misleading, as it would fail to capture the dynamics of the progressive evolution of risk factors over time. The U.K. Prospective Diabetes Study (UKPDS) Outcomes Model (2), the most widely used model in this field, incorporates both baseline and follow-up risk factors in equations to predict the likelihood of events. A further recent advance is the development of mortality scores in type 2 diabetes (3) in which a stepwise selection of predictor variables is used to develop a clinically meaningful prediction model. Although the investigators achieve a c statistic of 0.75 with their prediction model using up to 20 predictors, it is not clear whether the addition of predictor variables beyond a small number of established risk factors adds any clinically meaningful effect on outcome.
Of further concern, the Web-based calculator appears to give anomalous results, suggesting that when the level of some adverse risk factors is increased, the 6-year probability of survival also increases. For example, increasing BMI by 5 kg/m2 and blood pressure by 20/20 mmHg while keeping other parameters constant for a newly diagnosed diabetic patient, the overall survival appears to increase. These results are inconsistent with our current understanding of the effects of these risk factors on mortality. We would suggest the need for further validation of this statistical tool in comparison with others available.
Acknowledgments
No potential conflicts of interest relevant to this article were reported.