How Sweet Is It? The Use of Benchmarking to Optimize Inpatient Glycemic Control

Hospitals enrolled in the SHM Glycemic Control Mentored Implementation programs, or that subscribe to an electronic quality improvement program in glycemic control, upload blood glucose data in a secure and de-identified process. Only point-of-care (POC) blood glucose values are uploaded to prevent duplicate “mirror” serum and blood glucose values and to maximize ease of uploading data for participating hospitals. The blood glucose values, with their date, time, patient unit, and an encrypted numeric patient encounter identifier, are uploaded in a standardized format. Data from multiple months and units can be uploaded in a single session. Patient demographics, medications, sex, and other data fields are excluded. Patients with fewer than four blood glucose readings or with only one hospital day are excluded, and data are “scrubbed” for erroneous or questionable values.

Metrics summarizing rates of hyperglycemia, hypoglycemia, recurrent hypoglycemia, and the timeliness of hypoglycemia management and resolution were devised by SHM glycemia experts. SHM glucometrics use the patient-day as the preferred unit of analysis, a common practice with the advantage of providing a uniform unit of time, adjusting to some degree for repeated testing around glycemic excursions and other local variations in testing patterns.^13–18  The degree of glycemic control can be summarized as a day-weighted mean for a chosen group of patients (i.e., the mean of all readings for one patient-day, then averaged across all patient-days in that group), the percentage of patient-days with a mean (day-weighted) ≥ 180 mg/dl, the percentage of patient-days with all readings 70–179 mg/dl, or the percentage of patient-days with any blood glucose reading > 299 mg/dl. In a similar manner, hypoglycemia is summarized as the percentage of patient-days with at least one blood glucose reading < 70 mg/dl and severe hypoglycemia as the percentage of patient-days with any glucose reading < 40 mg/dl. Selected metrics are also expressed with the patient-stay as the unit of analysis, with a patient-stay being defined as consecutive days for which blood glucose readings are available for a patient (e.g., the percentage of patients with at least one hypoglycemic event during their stay, the percentage of hypoglycemic patients with recurrent hypoglycemic days during their stay, and the percentage of patients with a day-weighted mean ≥ 180 mg/dl over the course of their stay). The mean for a patient-stay is also based on a day-weighted methodology (mean glucose for day 1 + mean glucose for day 2 + mean glucose for day 3 / 3 days = a patient's mean glucose for a 3-day hospitalization) in a minor modification of Yale methodology.

After the data are uploaded, authorized users can run reports for their own institution on demand, rather than wait for fixed reports. Webinar demonstrations are available on the SHM website.¹⁹  Four types of reports are available from pull-down menus: a Tabular Summary Overview, Patient-Day reports, Patient-Stay reports, and a unique Hypoglycemia Management report, which tracks recurrent hypoglycemia and time intervals to repeat testing and resolution after the index event. All but the Tabular Summary Overview depict month-to-month results, as well as a summary row that provides the results for the entire time period. Menus allow for flexibility in report construction. For example, reports can be run for a single month or for all months in the database, and the performance for non-ICUs (or for ICUs) can be viewed as a composite, either separately for individual units or with any grouping of similar units. This variety of reports allows for establishing baseline performance, tracking trends over time, and establishing local benchmarking for similar units.

The Tabular Summary Overview report establishes the baseline level of inpatient glycemic control being achieved on any or all units. The baselines for hypoglycemia and hypoglycemia management are also available. The baseline report can raise awareness and support for glycemic control issues when presented to leadership and staff. For example:

The team at the institution in this example used the report to focus efforts on the timeliness and effective management of hypoglycemia. Nursing councils helped to simplify the hypoglycemia documentation and revised the hypoglycemia protocol to emphasize the identification and mitigation of contributors to the hypoglycemic event to prevent recurrent hypoglycemia. Educational programs with demonstration of nursing competency were initiated, and each unit received monthly feedback on progress until substantial improvement was achieved.

After baseline performance is established, it is possible to objectively assess the impact of interventions on baseline performance over time. As improvement teams educate staff and introduce interventions, increased awareness often leads to surveillance bias, with subsequent increases in voluntary reporting and anecdotes of hypoglycemic events. The availability of high-quality, objective data on hypoglycemia rates is a much better basis on which to make decisions about the safety of glycemic control interventions and can be reassuring to all concerned. The SHM reports offer run charts to complement tabular displays of month-to-month data because they are more accessible and intuitive to many audiences. Figure 1 is a run chart from the Patient-Day report, depicting a reduction in hypoglycemia over time.

Individual units within a given institution may vary significantly in their prioritization of insulin management principles, training, and processes related to glycemic control. The SHM reporting system can identify this variation by allowing for comparison of performance among similar units at a given institution. This internal bench-marking identifies top performers, encourages a healthy competition for better performance, and can lead to identification of best practices and improved buy-in for shared glycemic goals. Reports that depict unit-specific performance often resonate with staff more than institution-wide reports and can lead to better staff accountability for hypoglycemia management and other parameters.

External benchmarking services were repeatedly requested by hospitals involved in SHM's glycemic control improvement offerings because improvement teams wanted realistic goals and perspective on their performance. In 2013, SHM released its first round of external benchmarking reports to participating hospitals. “Non-core” areas such as dialysis units, recovery rooms, psychiatric wards, and emergency departments were excluded. All core units for each hospital were combined to establish the composite performance for the entire hospital, subdivided by ICU and non-ICU units. Each of the 76 hospitals received benchmarking reports depicting the performance for the past 6 months of uploaded data of every hospital, with a numeric code allowing them to identify their own hospital (but not other hospitals).

A tabular report similar to Table 1 allowed them to compare their own performance to the mean, median, and top-quartile performance levels of the 76 hospitals contributing non-ICU data. The wide range of performance is typical for hospitals with varied sophistication and standardization, as well as varied patient populations. The top quartile should be viewed as an “achievable benchmark,” rather than the ultimate goal because a full 25% of hospitals were able to achieve these levels across all of their non-ICU units on average.

Hospital performance is ranked for several important metrics, and a rank order bar chart plots absolute performance on the y-axis and consecutive relative performance on the x-axis. Figure 2 provides an example of the report for the percentage of patient-days > 299 mg/dl. Each hospital can distinguish itself from others in the cohort by entering the hospital numeric identifier in the benchmarking report, which high-lights its ranking in red.

Figure 3 is an example of an SHM benchmarking scatterplot. As in the rank order bar chart plots, each hospital can identify its own institution by entering its numeric coded hospital identifier, which highlights its institution in red. Each hospital's performance on a glycemic control parameter is depicted on the y-axis (in this case, the day-weighted mean blood glucose for non-ICU patients), while a hypoglycemia parameter (percentage of patient-days < 70 mg/dl) is depicted on the x-axis. Lower, median, and upper quartile boundaries are clearly marked.

This juxtaposition of hyper- and hypoglycemic performance on one graph makes the trade-offs and priorities for each institution more apparent. For example, in Figure 3, the highlighted hospital might be falsely reassured by looking solely at the glycemic control metric because it is in the top quartile with a day-weighted mean blood glucose of 148 mg/dl. However, its high rate of hypoglycemia (6.9% of patient-days) places it in the bottom quartile of hospital performance for this parameter. In this case, prioritizing hypoglycemia prevention would be paramount, perhaps leading to less stringent glycemic targets until this pressing safety issue is addressed. Note that good glycemic control and low hypoglycemia rates are not necessarily mutually exclusive, with some hospitals achieving top quartile performance on both at the same time, and many more achieving results better than the median for both.

External benchmarking can provide validation for interventions in areas in which the institution is doing well, reassure staff about the safety of glycemic control, motivate change in a prioritized and data-driven manner, and spur further local investigation, in addition to providing goals for improvement. External benchmarking results combined with internal benchmarking can be particularly effective in communicating with leadership and staff. For example:

Many hospitals trying to improve glycemic control find themselves without the capacity to gauge their performance or track changes over time. Hospitals building their own glucometrics may find it labor intensive and distracting from other improvement areas, and such efforts may still leave them without external benchmarking. In this article, we illustrate how SHM internal and external benchmarking tools can be used to improve inpatient control. Achievable benchmarks for a wide variety of glucometrics were established, while also revealing variability of performance across hospitals and opportunities for improvement. Presenting results in scatterplots reinforces the importance of weighing hyper- and hypoglycemic parameters together when assessing performance and setting priorities.

There are limitations to these tools. The hospitals enrolled are self-selected. Although we have attempted to streamline the data uploading process, it still requires effort on the part of each hospital. The repository of glucometrics data has to be accessed locally and converted to the comma-separated value (.csv) format, and each unit needs to be assigned descriptive labels defining care type (Critical Care, Non-Critical Care, or Other) and unit type (e.g., Medical, Surgical, Mixed, or Obstetrics/Gynecology). Once in a .csv format, the first upload generally takes no longer than 1–2 hours. Subsequently, the system recognizes the unit labels and classifies them automatically, and uploading subsequent months of data can generally be performed in 10–30 minutes. Participating sites do not need to do any additional work to receive external benchmarking reports, but they do need to gain some familiarity with the on-demand reporting engine to leverage it to maximal advantage. A few sites are unable to surmount the barrier of uploading data because of local concerns about confidentiality despite reassurances about how the data are protected and de-identified.

Only POC blood glucose data are used, despite potential accuracy issues, especially in the ICU setting. We made this decision because of the ubiquitous nature of POC blood glucose testing, the difficulty many hospitals would have uploading a combination of serum and POC blood glucose values, a desire to avoid duplicate blood glucose readings, and previous studies showing minimal impact on glycemic control parameters with exclusion of serum blood glucose readings.¹⁴ 

Although our measures are quite similar to others in use, there is currently no consensus on measurement methods.^1,9,13–18  We do not have access to medications, comorbidities, diagnostic codes, or demographics and thus cannot adjust for these factors. Use of the patient-day as the unit of measurement controls for some variation in testing patterns but does not entirely control for varied patterns of blood glucose testing. The SHM measurement tools do not fulfill the need to monitor insulin management patterns and order set utilization or to proactively identify and mitigate management issues with glycemic outliers. Improvement teams still need to address these needs locally.

On the other hand, the SHM benchmarking tools offer some advantages. Others offer tools for external benchmarking^17,18  or internal reporting and benchmarking,^14–16  but to our knowledge, SHM is the only source that offers the combination of internal and external benchmarking resources. The SHM tools also offer several unique features (e.g., run charts, flexible on-demand reporting, and metrics for recurrent hypoglycemia and the timeliness of hypoglycemia management). Customization of the reports down to the unit level is possible for internal benchmarking and tracking. Finally, the SHM glucometrics reporting engine is surrounded by other resources (e.g., an active online community, webinars, slide decks, and print materials) to assist improvement teams in addressing the full range of inpatient glycemic control issues. Future efforts will objectively evaluate the effectiveness of this combination of tools in improving hypo- and hyperglycemia.

The authors acknowledge the contributions of Ed Fink, MHSM, and Aaron Field of the University of California, San Diego Center for Innovation and Improvement Science, for their roles in creating the reporting and benchmarking capacity, along with collaborators from the SHM.