Response to Comment on Segar et al. Machine Learning to Predict the Risk of Incident Heart Failure Hospitalization Among Patients With Diabetes: The WATCH-DM Risk Score. Diabetes Care 2019;42:2298–2306

Identifying patients with type 2 diabetes mellitus (T2DM) at high risk for future heart failure (HF) has been challenging given the multisystem inputs that contribute to HF risk, inaccuracies in administrative coded data, and complexities with risk prediction models. In the machine learning–derived WATCH-DM (Weight [BMI], Age, hyperTension, Creatinine, HDL-C, Diabetes control [fasting plasma glucose], QRS Duration, MI, and CABG) score, we considered 147 candidate variables to create a simple, user-friendly, integer-based risk score to predict adjudicated incident HF events (1). We appreciate the critical appraisal of WATCH-DM by Fonseca and colleagues (2).

Our integer score was developed similarly to the well-established method popularized by the Framingham framework, in which the points associated with each level of each risk factor are relative to the points associated with an increase in age (3). Briefly, continuous variables were first converted to dichotomous variables. Cutoffs for the continuous variables were either determined by established guidelines (for example, the normal, overweight, and obese cutoffs for BMI) or by plotting the probability of outcome events against the numeric variable of interest using a locally weighted scatterplot smoothing (LOESS) function. As most developed risk scores are not routinely employed in clinical practice often due to perceived complexity and inconvenience, integer-based scores such as WATCH-DM may be more practical, user-friendly, and potentially more likely to be adopted.

We would like to thank Dr. Fonseca and colleagues for bringing to our attention the Building, Relating, Assessing, and Validating Outcomes (BRAVO) engine (4). In addition to simplifying prediction models to integer-based as a strategy to enhance potential clinical use as was our focus with the WATCH-DM project, we acknowledge and agree with the caveat noted by Fonseca and colleagues of the potential for increased penetrance and use of more complex risk-scoring algorithms when automated within the context of the standard electronic health care record, as is per their comments being pursued with the BRAVO models.

By including 17 separate risk equations, the BRAVO engine is able to estimate the risk of microvascular as well as macrovascular events in patients with T2DM. Conversely, our goal in creating the WATCH-DM risk score was to provide clinicians with a framework with three separate relationship modeling techniques that best suit their individual needs in identifying patients with T2DM at risk for heart failure. The WATCH-DM integer-based score is an easy calculation for clinicians to use at the bedside or in the clinic, while the regression-based score that optimizes model performance could be programmed for use in an electronic health care record. Finally, the machine learning–based risk score (using random survival forest modeling) is the most accurate method for predicting and stratifying patients at risk for heart failure. By making all three implementations of the WATCH-DM risk score publicly and freely available online (www.cvriskscores.com), our hope is that our tool can be useful for clinicians who are caring for patients with diabetes and thinking about what strategies can be used to help them.

Fonseca and colleagues also note that the use of electrocardiogram (ECG) parameters in a risk score may not be available in a primary care setting. Even though up to 95% of adult patients in the U.S. have an ECG within 30 days of their annual health examination (5), the benefit of using machine learning–based modeling in the WATCH-DM risk score is that random survival forests are able to handle missing data with minimal loss in accuracy (6). Thus, a patient does not have to have available ECG parameters to obtain an accurate 5-year risk of heart failure estimate.

Collectively, these risk scores highlight the complexity of initial risk prediction of HF events and the challenges with subsequent facile implementation of prediction tools in clinical practice. We are actively externally validating the WATCH-DM risk score in external cohorts and are implementing the score in multiple health care systems across the U.S.