An Automated Risk Index for Diabetic Ketoacidosis in Pediatric Patients With Type 1 Diabetes: The RI-DKA

Data were extracted on individuals with type 1 diabetes seen through the Endocrine and Diabetes Care Center at a large urban children’s hospital and academic medical center with three regional hospital campuses and seven ambulatory clinics; the cohort was drawn from the population of patients receiving care at any of these sites. We included all patients seen in fiscal years 2016 (FY2016) and 2017 (FY2017) who had an International Classification of Diseases, 9th or 10th revisions, Clinical Modification (ICD-9-CM or ICD-10-CM), diagnosis of type 1 diabetes; a diabetes duration ≥6 months; age ≤19 years; and at least one outpatient clinic visit and at least one A1C test in each fiscal year (FY2016 and FY2017) (Table 1). In FY2016, 1,864 patients met inclusion criteria, and in FY2017, 1,903 patients met criteria; 1,548 patients (81.3%) were included in both cohorts, with 355 new patients included in FY2017.

To develop the prediction model, possible predictors of DKA were first considered based on literature review (6–8) and the varied clinical expertise of a multidisciplinary team, comprising endocrinologists, advanced practice providers, certified diabetes care and education specialists, psychologists, social workers, data architects, and data analysts. An additional criterion was that predictors had to be able to be extracted automatically from the EMR. Seven variables were eventually selected for testing: patient age at the end of the fiscal year, sex, race/ethnicity, health insurance (public vs. private), most recent A1C value, occurrence of DKA in the prior two fiscal years, and insulin regimen (fixed mealtime dose vs. intensive insulin management [IIM] using an insulin-to-carbohydrate ratio and correction factor vs. insulin pump therapy). Outcome variables were obtained retrospectively from a late-binding enterprise data warehouse (EDW) developed for our institution that draws in data from the EMR and other institutional sources (e.g., financial claims) and a pediatric diabetes analytics application that is able to pull near real-time data from the EDW. The late-binding EDW architecture means that data are not linked to decision rules (such as a risk level cutoff for DKA in our model) until needed, which allows the database to change in response to potential changes in the model (e.g., if new risk factors are identified).

A1C values were obtained as part of routine care using the DCA 2000+ Analyzer point-of-care assessment (Siemens Healthcare Diagnostics, Deerfield, IL). We used the most recent A1C value (rather than an average) so that the index could update automatically whenever a new value was entered into the EMR. This also allows the index to be sensitive to recent changes in A1C, which may place a patient at increased risk. DKA was defined based on ICD-9-CM/ICD-10-CM diagnosis code in the hospital encounter (250.1x, 775.1, 277.09, 790.29), glucose ≥250 mg/dL (≥13 mmol/L), and pH level ≤7.30 and/or administration of intravenous insulin. DKA values were obtained from the preceding 2 years, as looking at only 1 year provided too few data points for analysis. DKA that occurred at diabetes diagnosis was excluded from analysis, as we reasoned that the causes of DKA would differ in newly diagnosed versus established patients. Other potentially important variables (e.g., family conflict and patient mental health) could not be included because of limitations of the data contained in a searchable format in our EMR. We considered using mental health diagnoses, but they are generally only entered into the EMR if a patient is among the subset being seen by the psychiatry or psychology services at our institution, so it was determined that these data would not be an accurate reflection of mental health conditions in our cohort.

Data from FY2016 were divided into training and validation datasets at a 70/30 split. Complete data from the next fiscal year (FY2017) provided the observation period for future DKA events and, as such, was used as an independent test dataset in a series of univariate logistic regression models. For nondichotomous variables, changes in odds were associated with each single-unit increase in the variable (e.g., an interval of 1 year for age and 1% for A1C). To guard against false positives, a Bonferroni-corrected significance level of 0.00625 was used to determine which variables from the univariate logistic regression model moved to the multivariable model. All analyses were conducted using IBM SPSS, v. 26, statistics software.

Four variables emerged as significantly associated with DKA in univariate analyses and were moved forward: A1C, race/ethnicity, type of insurance, and DKA in the past two fiscal years (Table 2). In fitting the multivariable model to the validation dataset, race/ethnicity and health insurance masked each other’s effect, so two reduced models with each of these variables were considered. The model with insurance type performed slightly better than the one with race/ethnicity and thus was moved forward. The final model comprised three variables: most recent A1C (odds ratio [OR] 1.29, 95% CI 1.16–1.45, P <0.0001), type of health insurance (OR 2.27, 95% CI 1.38–3.74, P = 0.0001), and prior DKA (OR 3.10, 95% CI 1.81–5.23, P <0.0001) (Table 3). As can be seen in Figure 1, the model correctly classified 77.6% of patients with regard to presence/absence of a subsequent DKA in FY2017.

This three-factor model was then used to create several point-based risk categories. We decided on a three-tier risk stratification system reflecting low-, moderate-, and high-risk categories, which maps onto widely used public health frameworks in pediatrics (16) and is similar to what was used for the RI-PGC. For A1C, a nonlinear point system was created, and the variable was rescaled and centered to zero to give credit to those with low A1C values. The point system was determined based on clinical judgment and allocated as follows: insurance: public/self-pay = 4, private = 0; prior DKA (last 2 fiscal years): yes = 5, no = 0; A1C: <8% = −2, 8.0–8.9% = 0, 9% = 0.5, >9% = +0.05, for every additional half percent up to 14%. The final risk score ranged from −2 to 10, and a three-tier system was implemented such that scores <2 were considered low risk, those 2–6 were moderate risk, and those >6 were high risk. The risk categories were also determined based on clinical judgment as follows: low: no risk factors (−2 to 0 points) or high A1C alone (+0.5 to 1 point); moderate: public insurance (+4 points) with or without a high A1C (+0.5 to 1 point) or prior DKA (+5 points) with private insurance and an A1C ≤8.5% (0 to −2 points); and high: prior DKA (+5 points) plus one other risk factor. As can be seen in Table 4, the risk categorization accurately identified patients at greatest risk for a subsequent DKA episode in the coming year. Most patients fell into the low-risk group, while a little more than one-third fell into the moderate-risk group, and ∼5–7% were identified as having high risk.

This article reports on an innovative, automated, EMR-based algorithm for predicting risk for DKA in established pediatric patients with type 1 diabetes. Scores are updated every time new relevant data such as an updated A1C result, a new episode of DKA, or a change in insurance are added to the EMR. This strategy allows the tool to provide continually updated risk assessments that can be used by providers to guide clinical care in real time. The results are logged in the EMR, and a chart can be brought up on screen quickly showing the current risk level, as well as 3-month and 1-, 2-, and 5-year trends. A next step in implementation will be to create a best practice alert that automatically identifies at-risk patients whenever their provider opens their chart.

Our findings converge closely with the results of another recent study examining DKA risk in a cohort of 1,428 patients with type 1 diabetes (17) that also found that a three-factor model including prior hospitalizations (for DKA, hyperglycemia, and ketonemia), A1C, and insurance status was the best predictor of DKA. That a similar model emerged from our data provides strong converging evidence for the importance of these variables in predicting DKA. Risk for DKA has also been found to be associated with insurance status in large-scale studies using the U.S. National Readmission Database (18) and the T1D Exchange clinic registry (19), and with higher A1C in multiple studies (8,19,20). Female sex has also frequently been found to be associated with DKA (6,19–21), so it is unclear why this was not a significant predictor in our study. It is possible that sex is not a significant factor by itself but only in interaction with other variables. Post-hoc analyses revealed potentially interesting interactions between sex and race/ethnicity in our cohort, but we did not have sufficient power to fully test these relationships. In addition, Rewers et al. (20) found that psychiatric disorders had a significantly larger effect on DKA risk in girls compared with boys. These potential interactions deserve further study in larger cohorts of patients.

A primary goal of risk identification and stratification is to guide allocation of scarce preventive intervention resources to the patients most in need of them. To this end, we are in the process of designing preventive interventions graded to patients’ level of risk. Universal diabetes education focused on DKA prevention has been shown to be effective in reducing DKA admission rates (22). Use of diabetes technologies such as insulin pump therapy and real-time continuous glucose monitoring (CGM) also has the potential to reduce the incidence of DKA in the diabetes population (23). A promising intervention targeted to the highest-risk patients is the Novel Interventions in Children’s Healthcare (NICH) program, which provides intensive services, including coordination of care, help with finding community resources, and training in problem-solving skills. Initial data suggest that NICH may be effective in reducing the incidence of DKA and health care costs in high-risk youth with type 1 diabetes (24).

Understanding the degree of risk is a first step toward risk reduction, but for intervention purposes, it is also important to understand the type of risk, and this is not currently well captured in our index. Youth may be more prone to DKA for very different reasons (e.g., because they omit insulin to control weight gain, because they do not understand appropriate sick-day management, because of mental health issues such as depression, or because their insurance has lapsed leaving them unable to fill their last prescription). More comprehensive psychosocial assessment of high-risk patients is likely to be crucially important for tailoring interventions to each patient’s needs.

One limitation to the risk index as currently designed is that it is unlikely to be accurate if patients do not have a recent A1C value in their chart (e.g., if they do not come to the clinic or go to a laboratory to get their A1C test). These patients may not have been captured in our cohort, which was limited to patients who had at least one clinic visit in each fiscal year analyzed. Missed clinic visits have been especially acute during the coronavirus disease 2019 pandemic, although more generally there is a known association between patients at highest risk for problematic diabetes-related outcomes and poor clinic attendance (25). Thus, there may be a cohort of high-risk patients who lack an updated risk score using our EMR-based tool. One option might be to use an estimated A1C based on mean blood glucose (26) or glucose management indicator (27), but these options would only be feasible for patients frequently checking blood glucose with a glucose meter or using CGM. Another option might be to use telehealth services to reduce the overall number of in-person visits required (e.g., by having patients go to an outside laboratory for their A1C test or use an at-home A1C test kit). There is evidence that at-home kits can accurately measure A1C levels (28), although there are logistic challenges to their use (e.g., mailing the kits to patients and having them returned), and they are frequently not covered by insurance. Still, some recent emerging evidence suggests that telehealth visits may help prevent DKA in higher-risk patients (29).

This study has a number of other limitations. First, the risk index was developed based on data from a single site. Although we were able to use a relatively large dataset drawing from a diverse patient population, it remains possible that different predictors would have emerged as significant in a different dataset drawing from a different sample. In addition, the test dataset was largely drawn from the same cohort as the training dataset, potentially inflating model performance. Our next step will be to externally validate the RI-DKA at other institutions and/or through large-scale diabetes databases. Another limitation is that the RI-DKA does not draw in psychological and mental health data (e.g., symptoms of depression), which have been shown to be an important additional contributor to DKA risk (21,30). To address this shortcoming, we plan to integrate data from the RI-PGC, the RI-DKA, and a screener for symptoms of depression that we have been using at our site (the Patient Health Questionnaire-9 [31]) to see whether predictive validity of the risk index can be further improved.

In conclusion, we have shown that risk for DKA can be estimated by an automated algorithm that draws from data easily available in the EMR and provides real-time support for clinical decision-making. More specifically, the risk index identifies patients with a significantly increased likelihood of experiencing an episode of DKA in the next year. Targeting these patients for preventive interventions has the potential to reduce the serious human and financial costs associated with this dangerous condition (9).

No potential conflicts of interest relevant to this article were reported.

D.D.S. helped to create the risk score, wrote the manuscript, researched data, and reviewed/edited the manuscript. R.B. researched data, helped to create the risk score, completed the data analysis, wrote the Research Design and Methods and results sections, and reviewed/edited the manuscript. S.U. researched data, wrote the introduction, and reviewed/edited the manuscript. M.V. researched data, contributed to writing the manuscript, and reviewed/edited the manuscript. K.R.H. and K.F.-B. led the high-risk team and efforts to create the risk score and reviewed/edited the manuscript. S.K.L., R.S., and S.K.G. reviewed/edited the manuscript. S.D.-T. also led the high-risk team, researched data, contributed to the Research Design and Methods section, and reviewed/edited the manuscript. D.D.S. is the guarantor of this work and, as such, had full access to all the data and takes responsibility for the integrity of the data and the accuracy of the data analyses.