A Comprehensive System for Identifying Patients With Type 1 Diabetes at Increased Risk for Diabetic Ketoacidosis at Texas Children’s Hospital

Texas Children’s Hospital, located in Houston, TX, is the largest pediatric hospital in the United States, with 973 inpatient beds and extensive outpatient clinics and services. It is the primary pediatric teaching hospital of Baylor College of Medicine. The Texas Children’s Endocrine and Diabetes Care Center is one of the largest pediatric endocrinology and diabetes centers in the country, with three inpatient facilities and seven ambulatory clinics. The service is staffed by a multidisciplinary team that includes endocrinologists, endocrine fellows, advanced practice providers, certified diabetes care and education specialists (CDCESs), dietitians, social workers, and consulting psychologists. Almost 500 youth with newly diagnosed type 1 diabetes are admitted to the hospital each year, with a total pediatric diabetes population of >3,400 patients.

Diabetic ketoacidosis (DKA) is an acute, severe, and preventable complication of diabetes that can result in significant morbidity and mortality. The most common cause of DKA in children and adolescents with established type 1 diabetes is inadvertent or deliberate insulin omission, often occurring in the context of chronically poor glycemic control. Identifying risk factors for diabetes-related morbidities is crucial to inform preventive efforts and target services to the patients in greatest need, with the ultimate goal of reducing the incidence of DKA.

Before the start of this project, our hospital had no structured, formal way to identify patients with type 1 diabetes at greatest risk for poor glycemic control and DKA. Patients considered high risk might have been referred to social work or psychology staff by their diabetes care providers, but this was a hit-or-miss process, with many patients likely being missed. More commonly, patients would only be referred after experiencing one or more episodes of DKA. The care of these patients was therefore mostly reactive, resulting in increased morbidity and hospitalizations and incurring significant expense. To reduce complications and improve population health for our patients with diabetes, we set out to develop a hospital-wide system for 1) identifying at-risk patients at the time of diabetes diagnosis and 2) providing an automatically updating risk assessment built directly into the electronic medical record (EMR) system that could potentially flag established patients with type 1 diabetes for point-of-care support. These risk assessments could then provide the basis for targeted preventive intervention.

In 2014, our team developed and validated the Risk Index for Poor Glycemic Control (RI-PGC), a psychosocial risk assessment tool designed to identify youth with newly diagnosed type 1 diabetes at heightened risk for poor glycemic control (defined as mean A1C ≥9.5%) and DKA (1). The RI-PGC is a nine-item scale designed to be administered as a brief structured interview at the time of diabetes diagnosis. It results in a single score that maps to low-, moderate-, and high-risk categories and translates into an estimate of increased risk that can be used to inform the type and intensity of follow-up care. The development and validation of the risk assessment at diabetes diagnosis has been described in detail elsewhere (1,2). Here we report on the integration of the RI-PGC into routine care and the development and implementation of a new tool, the Risk Index for Diabetic Ketoacidosis (RI-DKA), to estimate risk for DKA in established patients with a diabetes duration ≥6 months, as part of a broader quality improvement (QI) initiative to improve diabetes care at our institution.

Baseline data were obtained through a late-binding enterprise data warehouse (EDW) developed for Texas Children’s Hospital that integrates data from the EMR system and other internal sources, and a pediatric diabetes analytics application that pulls nearly real-time data from the EDW. Data available in the EDW include clinical variables such as A1C and treatment modalities, number of clinic visits and hospitalizations, demographics, and insurance and claims data. The late-binding architecture allows for these data to be mapped onto decision rules (e.g., a risk-level cutoff for DKA) more flexibly than traditional data warehousing, providing the ability to make quick changes to the model if new information (e.g., newly identified risk factors) becomes available. These tools are a key component of ongoing efforts to improve diabetes care at our institution.

In 2015, when this project was initiated, the prevalence of DKA in our population with established type 1 diabetes was 9.6%, and nearly one in five incidents of DKA (19.6%) were repeat episodes. These data led us to realize that allocating resources to prevent recurrent DKA could be an especially efficient and effective way to positively affect the health of our patients with type 1 diabetes. This QI project was therefore initiated with the specific aim of identifying patients at highest risk for DKA, including recurrent DKA, who could then be targeted for preventive intervention, with the ultimate goal of reducing the overall incidence of DKA in our population with established type 1 diabetes to <5%.

This project began in January 2015, when the Texas Children’s Endocrine and Diabetes Care Center formed a multidisciplinary QI Care Process Team (CPT) to develop strategies to improve support and care for patients with diabetes. Regarding the specific QI initiative described in this article, the first aim (integrating the RI-PGC into standard care) ran from March to December 2015; the second aim (developing and validating the RI-DKA) ran from March 2015 through August 2018; and the third aim (implementing the RI-DKA into the EMR system) ran from August 2018 to April 2019. Efforts to better integrate these indexes into daily clinical practice continue to this day.

QI efforts have long been one of the major pillars of care at the Texas Children’s Endocrine and Diabetes Care Center. The multidisciplinary Diabetes CPT is subdivided into five workgroups with different areas of focus (Clinic, Community, Inpatient, Education, and High-Risk), all working together to improve the care we provide to children with diabetes. The High-Risk CPT, which initiated the project reported here, is co-led by a pediatric endocrinologist and a social worker, and is composed of endocrinologists, advanced practice providers, CDCESs, psychologists, social workers, data architects, data analysts, and parents/family members of patients.

The High-Risk CPT meets biweekly to discuss and evaluate the department-wide QI initiatives for our diabetes cohorts. The staffing of the Endocrine and Diabetes Care Center has also evolved as a result of the needs identified by the QI initiatives, including adding more social workers and psychologists to our team to improve the care of patients with high-risk diabetes. This strategy proved especially important to managing our workflow, as risk screening resulted in increased referrals to social work and psychology staff.

The first step of this QI project was to integrate the RI-PGC into standard clinical care for all patients newly diagnosed with type 1 diabetes. After extensive discussion, the team agreed that the RI-PGC would be administered by social workers within the diabetes service. The social workers believed that using the RI-PGC would have a negligible impact on their workflow, as it merely formalized some of the questions they were already asking as part of their standard assessments. To ensure a smooth integration into routine care, we completed a series of Plan, Do, Study, Act (PDSA) cycles with the following specific, measurable, applicable, realistic, and timely (SMART) aim: The RI-PGC will be completed as part of standard clinical care and documented in the EMR for ≥90% of patients newly diagnosed with type 1 diabetes by December 2015.

PDSA 1 focused on developing standard operating procedures for social work implementation. This process involved adapting our prior procedure (which used Psychology trainees) for use by social work staff, through discussion with all key stakeholders. This step was completed in August 2015.

PDSA 2 focused on training social workers to administer the RI-PGC, using a standardized training framework (3). This step was completed in September 2015.

PDSA 3 involved integrating the RI-PGC into the EMR system. This process involved developing a flowsheet for diabetes social workers that allowed for direct entry of item responses into the EMR and automatic generation of the risk score. This step was completed in November 2015.

PDSA 4 involved social workers integrating the RI-PGC into standard care. This process involved administration of the tool on the inpatient unit, as well as a procedure for follow-up for positive screens. Specifically, for all patients with an RI-PGC score ≥4 (indicating high risk for problematic outcomes), social work staff would follow-up with a more comprehensive assessment and possible referral for psychology services.

Our next series of steps was focused on developing an analogous risk-screening tool and procedure for established patients with a diabetes duration ≥6 months. In this group, it was decided to focus solely on DKA as the most relevant outcome, given the outsized impact of DKA on population health. In a series of regression analyses of data from our EMR system, three variables emerged as significant predictors of DKA: most recent A1C, type of health insurance (public vs. private), and prior occurrence of DKA in the past 2 years (yes/no). These three factors feed into a global risk score that maps onto categories characterized as low, moderate, and high risk. Preliminary data suggest that the risk categories discriminate very well between patients who do and do not go on to experience DKA in the coming year. The formal development and validation of the RI-DKA will be reported elsewhere (4). Below, we focus on the implementation of the index into the EMR system and routine care.

Our initial discussions focused on how to administer the RI-DKA. First, we considered in-person administration in the same manner that was used for the RI-PGC, but we realized this would not be feasible for screening the thousands of patients we see each year, especially given the time required for administration, scoring, and documentation. Faced with a similar problem, some hospitals have begun using automated risk-scoring systems for adverse events built directly into their EMR system, with the goal of reducing burden on clinical staff and decreasing inaccuracies associated with human error (5,6). We decided to create an EMR-based tool that would generate a risk score for DKA based on variables that could be automatically extracted from data available in each patient’s chart.

We worked with our hospital’s Information Services (IS) department to build the index into the EMR system. The system was designed so that each time a patient’s data are updated, such as when a new A1C value is entered, an updated risk score is generated and becomes part of the patient’s diabetes flowsheet. The score is color-coded for easy discernment of risk level. This tool includes a temporal graph with the score ranges for each risk category indicated on the y-axis so providers can see how the scores and risk levels change over a user-defined period of time. The three variables are presented next to the graph so providers can see what specific factors are contributing to the patient’s current risk and whether they have changed as well (Supplementary Figure S1).

Once the electronic tool was created, our SMART aim was to have an RI-DKA risk score generated in the EMR for at least 90% of established patients by 30 June 2017. We reached this goal in early 2019. This was later than planned, due to changing definitions of the risk score during the development phase and delays in the build given the multiple priorities of the hospital-wide IS department. Several additional factors led to delays in using the risk score clinically. After the score was built into the EMR system, the co-lead of the High-Risk CPT at that time left our institution, and, soon afterward, the coronavirus disease 2019 (COVID-19) pandemic hit, leading us to pause substantial changes to standard practice as we adapted our model of care to the pandemic and switched to mostly telemedicine appointments. We also had significant staffing challenges at that time with regard to social work capacity (caused by factors unrelated to this project). Once we returned to full social work staffing, we were able to resume working with the risk score on a broader scale.

When RI-PGC implementation first went live in November 2015, social work staff were able to complete the screening for 75% of patients. Initial feedback from our social workers indicated that the tool was easy to use and did not significantly change their workflow; the 75% completion rate had more to do with preexisting factors (e.g., not having enough social workers) and was comparable to the assessment rate before using the risk index. Instead, use of the risk index helped standardize the process and provided an easily understood score that could be used by different providers. Through iterative problem-solving, we were able to reach our goal of having an RI-PGC score documented in the EMR for ≥90% of newly diagnosed patients by the end of December 2015.

Within a brief period after the RI-DKA had been built into the EMR system, 98% of established patients with type 1 diabetes had a documented risk score in the EMR. Missing scores were the result of missing variables, such as when there were no available A1C test results within the past year. Use of the risk index has not changed social work assessment processes themselves, as social workers follow up higher scores with the more comprehensive assessment that has long been part of their standard of care.

As noted, there have been delays in formally rolling out the risk index to clinical providers, although most are aware that the index is available in the EMR system. Preliminary feedback from a few of the providers who have been using the index indicates that the score is very helpful in identifying patients at high risk for DKA who might need additional support. These providers also appreciate the visual display of the DKA risk trend over time, which was noted to be useful as an index of progress or increasing concern.

However, providers also noted barriers to finding and using the score. Some specialties (e.g., CDCES) did not have access to the score, although we are now working with IS to rectify this problem. Providers also noted that the score is located in an area of the EMR outside of the standard workflow, so when clinics are busy, they often do not have time to look for the score. This issue was recently addressed by adding the risk score to a pre-clinic planning document that identifies all patients scheduled for clinic that day and indicates whether they are due for updated laboratory testing or standard-of-care assessments by one of the social workers, dietitians, or CDCESs. The risk score was also added to the “diabetes passport,” a clinical summary sheet and checklist given to clinical staff on the day of a patient’s appointment that is used to ensure that all needed services identified in the pre-clinic planning document are provided. We do not yet have data on the impact of the risk scores on referrals for social work services, but in anticipation of a likely increase in such referrals, we have added more social workers to our team.

We are currently working with one of the EMR specialists at our institution to make the risk score more visible within our standard workflow. Once that step is completed, we will schedule trainings with clinical staff in the Section of Pediatric Diabetes and Endocrinology to provide a formal introduction to the risk scores, including how to find them, interpret them, and view trends in the scores over time. We will also present information on referring higher-risk patients for intervention. Once we are able to complete a formal presentation, we plan to start collecting data on clinician usage and satisfaction.

We are also considering putting into place a “best practice alert” (BPA) that pops up on the EMR screen when a clinician opens a high-risk patient’s chart, further automating the process. Implementation of the BPA was halted during the height of the COVID-19 pandemic, as EMR staff were pulled off of ongoing projects to assist with the hospital’s urgent telemedicine needs. We are also debating how best to implement the BPA given that there are already so many BPAs built into the EMR system, and “alert fatigue” is common.

As noted, the long-term goal of this project is to reduce the incidence of DKA in our population with established diabetes. It will be important to see whether implementation of risk prediction into standard care by itself has an impact on DKA incidence (e.g., by heightening providers’ awareness of the greater vulnerability of some of their patients). The High-Risk CPT has also been working on other initiatives to improve care for children at high risk of adverse events. We plan to tailor some of these initiatives even more closely based on patients’ risk scores. These include, but are not limited to 1) an “Extra Care” cohort with a dedicated social worker, to provide more personalized psychosocial care to high-risk patients; 2) comprehensive social work and psychology assessments for moderate- and high-risk patients; and 3) standard-care diabetes education and brief social work and psychology assessments for low-risk patients. The Extra Care cohort has already been implemented with a subset of the highest-risk patients. Social work assessment and diabetes education have long been part of standard care, but the risk scores are allowing us to tailor these interventions more closely to patients’ assessed level of need. These interventions will be described in detail in subsequent articles.

Incorporating universal risk screening into standard clinical care in a large hospital can be a daunting task. The methods must be feasible, acceptable to a wide array of stakeholders, cost-effective, and well integrated into routine practice without disrupting clinical operations. Implementation would have been impossible without the close input and collaboration of the social workers who administer the RI-PGC and who were able to seamlessly merge the process into routine care on a very busy inpatient service. Support from leadership helped to ensure that potential barriers to making changes to longstanding policies and procedures were minimized. Finally, having data analysts and information technologists on the team was crucial to ensure that we understood what the EMR system was capable of, allowing us to implement a risk-prediction algorithm that automatically generates output, is user-friendly, and provides point-of-care support for health care providers and their patients.

No potential conflicts of interest relevant to this article were reported.

D.D.S. wrote the manuscript, researched data, and reviewed/edited the manuscript. M.V. and S.U. researched data, contributed to writing the manuscript, and reviewed/edited the manuscript. K.R.H. and K.F.-B. led the high-risk team and efforts to create the risk score and reviewed/edited the manuscript. S.K.L., R.S., and S.K.G. reviewed/edited the manuscript. R.B. researched data, completed the data analysis, and reviewed/edited the manuscript. S.D.-T. also led the QI project, researched data, contributed to writing the manuscript, and reviewed/edited the manuscript. D.D.S. is the guarantor of this work and, as such, takes responsibility for the accuracy and integrity of the content.