Accuracy of the GlucoWatch G2 Biographer and the Continuous Glucose Monitoring System During Hypoglycemia: Experience of the Diabetes Research in Children Network

The study was conducted by the Diabetes Research in Children Network (DirecNet). The study protocol, statistical methods, and informed consent procedures have been described in prior publications (18,19) and are briefly summarized herein.

The major eligibility criteria for the subjects included age 1–18 years, a clinical diagnosis of type 1 diabetes, and insulin therapy (either a pump or injections) for at least 1 year. Each subject used the GW2B and CGMS during a 24-h clinical research center (CRC) admission. To assess CGMS function over the entire 72 h of its life span, one-third of the subjects had the CGMS inserted 48 h before admission, one-third 24 h before admission, and one-third on admission to the CRC. Subjects were offered the option of wearing a second CGMS during the inpatient stay. After admission to the CRC, a GW2B sensor was placed and calibrated. A second GW2B sensor was placed at a later time so that there would be a minimum 2-hour overlap between the two GW2Bs and such that at least one GW2B would be functioning for the 24 h of serum glucose measurements. A One Touch Ultra Meter (Lifescan, Milpitas, CA) was used to obtain glucose measurements for calibrating the sensors.

Blood samples were obtained hourly during the day, every 30 min overnight, every 5 min for up to 90 min during an insulin-induced hypoglycemia test, and at additional times when the sensors were calibrated or if there were symptoms of hypoglycemia. Glucose measurements were made at the DirecNet Central Biochemistry Laboratory at the University of Minnesota using a hexokinase enzymatic method.

Subjects ≥7 years of age and of sufficient weight to accommodate extra blood sampling underwent an insulin-induced hypoglycemia test. The purpose of the hypoglycemia test was to assess sensor function during an acute fall in glucose levels into the mildly hypoglycemic range. If the pretest blood glucose was <80 mg/dl, juice or other carbohydrate was given orally to raise the blood glucose above this level before starting the test. For the test, 0.05–0.10 units regular insulin/kg body wt were given by intravenous bolus injection. After 30 min, a second dose could be given if the target glucose (<55 mg/dl) had not been achieved. For subjects who did not reach a glucose level <55 mg/dl, the test ended after 90 min. For the subjects whose glucose decreased to <55 mg/dl, sampling continued every 5 min after hypoglycemia treatment until the glucose level was >80 mg/dl. Treatment of hypoglycemia was with either oral or intravenous glucose at the discretion of the investigator.

For analysis, the GW2B glucose values were adjusted for a 17.5-min lag time and matched to reference serum glucose measurements drawn within ±5 min of the sensor reading except during the insulin-induced hypoglycemia test where the matching was within ±2.5 min. The CGMS glucose values were matched to reference serum glucose measurements drawn within 2.5 min of the sensor reading after adjusting for a CGMS lag time of 2.5 min. The lag times for the GW2B and CGMS principally represent the time involved to sample the interstitial fluid and measure the glucose. Thus, the GW2B value, when it appears on the device, represents the glucose level 15–20 min earlier.

The absolute difference was defined as the absolute value of sensor glucose value minus the reference glucose value. Sensitivity was defined as the percentage of reference values ≤60 mg/dl in which the sensor was also ≤60 mg/dl. The false-alarm rate was calculated as the percentage of sensor values ≤60 mg/dl in which the reference value was >60 mg/dl. During the course of the study, Medtronic MiniMed modified the sensor fabrication process that had been used since 1999. Accuracy analyses were conducted separately for the “original” and “modified” sensors. Statistical comparisons of the original versus modified CGMS sensors were conducted using the bootstrap (a resampling technique to determine the statistical margin of error) (20).

Hypoglycemic episodes occurring overnight (11:00 p.m.–6:00 a.m.) were defined as periods with at least two sensor glucose readings ≤60 mg/dl and no readings >70 mg/dl (the episode was considered to have ended when the glucose reading was >70 mg/dl). Episodes were required to be separated by a period of at least 30 min with all sensor readings >70 mg/dl. Only episodes during which there was at least one reference glucose measurement were counted. The episode was considered to be confirmed if there was at least one reference glucose value during the period that was ≤70 mg/dl and not confirmed if all reference values were >70 mg/dl.

The study included 91 subjects ranging in age from 3.5 to 17.7 (mean 9.9) years; 51% were female and 85% were Caucasian.

The reference glucose was ≤60 mg/dl for 192 of the 3,672 GW2B reference glucose paired values. For the 192 reference glucose values ≤60 mg/dl, the median absolute difference of the GW2B values was 26 mg/dl; 31% of sensor values were within 15 mg/dl of the reference value.

Results were similar for hypoglycemic reference values compared with GW2B values obtained during the intravenous insulin test and those obtained at other times (Table 1). Among 45 subjects undergoing the intravenous insulin test, the reference glucose nadir was ≤60 mg/dl in 34 subjects wearing 48 GW2Bs during the test. Of these, the GW2B nadir was also ≤60 mg/dl for 12 (25%) case subjects, 61–80 mg/dl for 27 (56%), 81–100 mg/dl for 8 (17%), and >100 mg/dl for 1 (2%). Among the 11 subjects wearing 16 GW2Bs in whom the reference glucose nadir was >60 mg/dl, the GW2B nadir was ≤60 mg/dl in 1 of the 16 (6%).

For a hypoglycemia alarm setting of 60 mg/dl, GW2B sensitivity for detection of an actual serum glucose level ≤60 mg/dl (based on the reference glucose value) would be 23%; 51% of alarms would be false. The reference glucose was >80 mg/dl for 18% of these GW2B alarms ≤60 mg/dl. As can be seen in Table 2, greater sensitivity for detecting a true glucose value ≤60 mg/dl would be achieved by raising the alarm setting, but this is at the expense of progressively higher false-alarm rates.

During overnight monitoring (11:00 p.m.–6:00 a.m.), the GW2B reported 21 hypoglycemic episodes occurring in 16 subjects. A reference glucose value was obtained during 18 of the episodes. Hypoglycemia (reference value ≤60 mg/dl) was confirmed in 10 (56%) of the 18 episodes. In the eight nonconfirmed episodes, the lowest reference glucose value ranged between 75 and 108 mg/dl.

The reference glucose was ≤60 mg/dl for 401 of the 6,778 CGMS reference glucose paired values. For the 401 reference glucose values ≤60 mg/dl, the median absolute difference of the CGMS values was 19 mg/dl; 42% of sensor values were within 15 mg/dl of the reference value. There were 356 CGMS reference glucose pairs from original CGMS sensors and 45 pairs from modified sensors. The median absolute difference was 20 mg/dl for the original sensors compared with 15 mg/dl for the original sensors (P = 0.09).

Results for hypoglycemia occurring during the intravenous test and at other times are presented in Table 1 for the original and modified sensors. During the intravenous insulin test, the reference glucose nadir was ≤60 mg/dl in subjects using 51 original sensors and three modified sensors. For the 51 original sensors, the sensor nadir was ≤60 mg/dl in 26 (51%), 61–80 mg/dl in 11 (22%), 81–100 mg/dl in 12 (24%), and >100 mg/dl in 2 (4%). For the three modified sensors, the sensor nadirs were 42, 48, and 77 mg/dl. Among the 10 subjects wearing 14 original CGMSs in whom the reference glucose nadir was >60 mg/dl, the CGMS nadir was ≤60 mg/dl in 2 of the 14 (14%). Among the six modified CGMSs worn by three subjects in whom the reference glucose nadir was >60 mg/dl, the CGMS nadir was also >60 mg/dl in all six.

Because the CGMS is retrospectively calibrated, real-time sensitivity for detection of hypoglycemia cannot be assessed. However, we evaluated the data as if the same level of accuracy would be present with prospective calibration. For an alarm setting of 60 mg/dl, the sensor sensitivity for detection of a glucose level ≤60 mg/dl (based on the reference glucose value) would be 36% for the original sensors and 49% for the modified sensors (P = 0.37), and 63 and 58%, respectively, of alarms would be false. The reference glucose was >80 mg/dl for 39 and 15% of these original and modified CGMS alarms ≤60 mg/dl, respectively. As with the GW2B, greater sensitivity for detecting a true glucose value ≤60 mg/dl is achieved by raising the alarm setting, but this is at the expense of a higher false-alarm rate (Table 2).

During overnight monitoring (11:00 p.m.–6:00 a.m.), the CGMS reported 30 hypoglycemic episodes (as defined in research design and methods) occurring in 25 subjects with original sensors and four episodes in 4 subjects with modified sensors. For 26 of the episodes with the original sensors and for three of the episodes with a modified sensor, a reference glucose value was obtained during the episode. Hypoglycemia (reference value ≤60 mg/dl) was confirmed in 8 (31%) of the 26 events detected by the original sensors and in all three detected with the modified sensors. For 12 of the 18 unconfirmed episodes with the original sensors, the lowest reference glucose value was >100 mg/dl.

This study was designed to evaluate the accuracy of the GW2B and CGMS in children and adolescents with type 1 diabetes. Because a critically important function of these sensors is in the detection and prevention of hypoglycemia, the current report focuses on performance of the devices during both induced and spontaneous reductions in serum glucose concentrations. Our results demonstrate that neither the GW2B nor the CGMS is accurate with respect to reporting glucose values in the hypoglycemic range. For reference serum glucose values that were ≤60 mg/dl, the median absolute differences of the GW2B and the original and modified CGMS sensors were 26, 20, and 15 mg/dl, respectively, and only 31, 41, and 51% of the sensor values, respectively, were within 15 mg/dl of the reference glucose values. Additionally, reference glucose levels did not confirm a substantial fraction of the low sensor values reported by both systems. As we have reported in detail elsewhere (18,19), the accuracy of the GW2B and the CGMS is considerably better when reference glucose levels are >100 mg/dl.

A major benefit of real-time compared with retrospective glucose sensing is the capacity to equip the system with hypoglycemia alarms. Consequently, we also explored the sensitivity and specificity of a range of alarm settings for the detection of hypoglycemia using the GW2B and reference serum glucose values. However, it must be remembered that the glucose value appearing on the device is estimating the serum glucose level 15–20 min earlier. As indicated in Table 2, the GW2B alarm would have to be set at 120 mg/dl to capture at least 90% of actual serum glucose concentrations that were ≤60 mg/dl. However, at this setting, 85% of alarms would be false positives. Setting the alarm to trigger at a sensor value of 60 mg/dl would reduce the false-alarm rate to 51%, but only 23% of true low reference glucose levels would be identified. For hypoglycemic episodes detected by the GW2B overnight, only 10 of 18 were confirmed by a reference serum glucose obtained during the episode. A previous GW2B accuracy study in 66 children reported that with an alarm setting of 90 mg/dl, sensitivity was ∼90% to detect a reference glucose level ≤70 mg/dl, but the false-alarm rate was ∼70% (11). In our data, an alarm level of 90 mg/dl would detect 72% of values ≤70 mg/dl with a false-alarm rate of 58%. In addition to its hypoglycemia alarm function, the GW2B has a “down-alert” function that alarms when the trend of sensor values indicates impending hypoglycemia. We are in the process of developing an analytic approach to assess this function of the GW2B.

The CGMS does not provide data in real time: glucose values are calculated using calibration values from both before and after the glucose sampling. Nevertheless, we evaluated how well the CGMS would alarm for hypoglycemia under the assumption that the accuracy would be similar in real time. It is likely that accuracy would be lower with prospective calibration, so these results should be considered to be the best-case scenario for the CGMS. Similar to our findings with the GW2B, sensitivity was relatively low with an alarm setting of 60 mg/dl, although better with the modified sensors than with the original sensors. As with the GW2B, setting a higher alarm setting increased sensitivity but at the expense of a high false-alarm rate.

For hypoglycemic episodes detected overnight by the original CGMS sensors, only 8 of 26 were confirmed by a reference serum glucose obtained during the episode. Although there were only three nocturnal episodes of hypoglycemia with the new sensors, all were confirmed by the reference glucose measurements. Several studies that have used the original CGMS sensors to evaluate metabolic control in children and adolescents with type 1 diabetes have reported an unexpectedly high incidence of asymptomatic hypoglycemia, especially at night (12–14). McGowan et al. (15) recently examined CGMS function at night in seven children with type 1 diabetes. In that study, four of five nocturnal hypoglycemic episodes that were detected by the CGMS could not be verified by reference glucose measurements. Guerci reported the CGMS’s sensitivity to detect hypoglycemia to be 33% (similar to what we found using original sensors) (21). Our data support the contention that previous clinical studies using the CGMS in children with type 1 diabetes overestimated the true incidence of hypoglycemia.

Because the GW2B and CGMS are more accurate in sensing high versus low glucose values, (18,19) the systems are likely to be of value in adjusting bolus and basal insulin doses in individuals with elevated HbA_1c levels. For the GW2B, the accuracy for low glucose values limits the feasibility of its alarm function. Whereas some users will accept a high false-alarm rate to detect a high proportion of low glucose values, many will not. Thus, for many users, the greatest value of the GW2B may be for detecting trends and not for serving as a sentinel for hypoglycemia. The accuracy of this early generation of glucose sensors is reminiscent of the early generations of glucose meters, which were less accurate than those currently available. Therefore, an expectation exists that future generations of sensors will have improved accuracy resulting in greater use for the detection of hypoglycemia.

Eva Tsalikian, MD; Roy W. Beck, MD, PhD; William V. Tamborlane, MD; H. Peter Chase, MD; Bruce A. Buckingham, MD; Stuart A. Weinzimer, MD; Nelly Mauras, MD; Katrina J. Ruedy, MSPH; Craig Kollman, PhD; and Dongyuan Xing, MPH.

This is listed in alphabetical order with clinical center name, city, and state. Personnel are listed for Principal Investigator (PI), Co-Investigator (I), and Coordinators (C). 1) The Barbara Davis Center for Childhood Diabetes, University of Colorado, Denver, Colorado: H. Peter Chase, MD (PI); Rosanna Fiallo-Scharer, MD (I); and Jennifer H. Fisher, ND, RN (C). 2) The Department of Pediatrics, University of Iowa Carver College of Medicine, Iowa City, Iowa: Eva Tsalikian, MD (PI); Michael J. Tansey, MD (I); and Linda F. Larson, RN (C). 3) The Nemours Children’s Clinic, Jacksonville, Florida: Tim Wysocki, PhD, ABPP (PI); Nelly Mauras, MD (I); Kristen M. Gagnon, MS, RD (C); Pauline Todd, RN (C). 4) The Division of Pediatric Endocrinology and Diabetes, Stanford University, Stanford, California: Bruce A. Buckingham, MD (PI); Darrell M. Wilson, MD (I); Jennifer M. Block, RN, CDE (C); and Elizabeth L. Kunselman, RN, CDE (C). 5) The Department of Pediatrics, Yale University School of Medicine, New Haven, Connecticut: William V. Tamborlane, MD (I); Stuart A. Weinzimer, MD (PI); and Elizabeth A. Doyle, MSN (C).

The Jaeb Center for Health Research, Tampa, Florida: Roy W. Beck, MD, PhD; Katrina J. Ruedy, MSPH; Craig Kollman, PhD; Dongyuan Xing, MPH; Pamela S. Moke, MSPH; and Lara M. Labastie.

Dorothy M. Becker, MBBCh; Christopher Cox, PhD; Christopher M. Ryan, PhD; Neil H. White, MD, CDE; and Perrin C. White, MD.

Michael W. Steffes, MD, PhD; Jean M. Bucksa, CLS; and Maren L. Nowicki, CLS.

Gilman D. Grave, MD; Barbara Linder, MD, PhD; and Karen K. Winer, MD.

Appreciation is expressed for the work also performed by the CRC nurses at the five clinical centers.