Blood glucose monitors are an important tool for people with diabetes to assess their physiological status and allow them to properly dose themselves with medication or food. The need to adjust the amount of treatment is greatest when the blood glucose level is in the hypoglycemic range because of the risk of acute brain damage if the condition is not quickly treated. A variety of technologies are available to assist patients with detecting hypoglycemia. How accurate are the various types of technologies at detecting hypoglycemia? Are the new continuous glucose monitoring technologies as accurate as current performance guidelines for home blood glucose monitors specify?
In this issue of Diabetes Care, the Diabetes Research in Children Network (DirecNet) collaborative study group concludes (1) that the GlucoWatch G2 Biographer (GW2B) and continuous glucose monitoring systems (CGMSs) do not reliably detect hypoglycemia in children and adolescents. They noted that as few as 31% of GW2B values and as few as 42% of CGMSs (including both first- and second-generation sensors) were within 15 mg/dl of reference serum values. When a hypoglycemia alarm value was set at 60 mg/dl, the GW2B demonstrated that the sensitivity to detect hypoglycemia was 23%, the specificity to detect hypoglycemia was 49%, and the false-alarm incidence was 51%.
The two continuous glucose monitors probably performed less accurately in this DirecNet study than many clinicians would expect of new technology that has been approved by the U.S. Food and Drug Administration within the past few years. The performance statistics for glucose monitoring systems that are frequently quoted from the literature or from the package inserts of manufacturers mostly derive from studies that measure patients whose blood glucose levels can be low, normal, or high. Because most patients spend little time in a state of hypoglycemia, the majority of data points in these studies derive from the normoglycemic or hyperglycemic ranges. Data from the DirecNet study (1) and from older studies (2,3) suggest that the performance of blood glucose monitoring systems would be best evaluated if separate performance goals for hypoglycemic, normoglycemic, and hyperglycemic ranges were established. Current glucose monitoring systems, including home blood glucose monitors and continuous glucose monitors, such as the GW2B and CGMS, do not perform as accurately in the hypoglycemic range as they do in higher ranges.
Five sets of criteria for blood glucose monitor performance are frequently referred to in the medical literature. They are each identified by the professional or regulatory agency that proposed them. Criteria proposed by the National Committee for Clinical Laboratory Standards (4), the American Diabetes Association (ADA) in 1987 (5) and again in 1996 (6), and the U.S. Food and Drug Administration (7) are stratified into two parts according to whether the blood glucose level is either <100 or ≥100 mg/dl. Criteria proposed by the International Organization for Standardization (ISO) (8) are stratified for blood glucose levels <75 or ≥75 mg/dl. These criteria were all developed before any continuous glucose monitors became available on the market, and it is possible that eventually alternate performance standards for glucose monitors may become developed (9). None of these agencies’ criteria, however, are stratified for the performance of blood glucose monitors during hypoglycemia, which is usually defined as a blood glucose level <50–70 mg/dl (10,11).
Consumers and health care professionals need to know the performance of their blood glucose monitors, and performance information is especially important for a patient to trust that their monitor will detect hypoglycemia. It is necessary to understand the likelihood that hypoglycemia will be identified when this condition occurs as well as the likelihood that a hypoglycemic reading truly reflects a physiological state of glucose depletion in the blood, as opposed to the low number being a “false-alarm” low reading. Diagnosing hypoglycemia is the most important function of home blood glucose monitoring because this problem, if detected, can be immediately and effectively treated, preventing acute catastrophic end-organ damage. Specific performance goals for blood glucose monitors during hypoglycemia are needed because blood glucose monitors do not perform consistently over the physiological range of blood glucose levels.
Evidence demonstrating the different performance of blood glucose monitors in the hypoglycemic, normoglycemic, and hyperglycemic ranges comes from studies on three different types of monitoring systems. The three types of monitors that provide virtually all currently measured home blood glucose values are 1) episodic home blood glucose monitors (multiple manufacturers), 2) the GW2B (Cygnus, Redwood City, CA), and 3) the CGMS (Medtronic MiniMed, Northridge, CA). For all three types of monitoring systems, performance is the worst in the hypoglycemic range. Thus, even when acceptable global standards are achieved by a monitoring system, the distribution of readings from the three glycemic categories can affect the overall percentage of readings that are within the target range. This diluting effect on global performance occurs because hypoglycemic readings are less often within target ranges than are readings from the euglycemic and hyperglycemic ranges. A set of data that is poor in hypoglycemic readings is more likely to achieve global performance standards than a set that is rich in hypoglycemic readings.
Home blood glucose monitors, identical or similar to currently available monitors, have been recently evaluated according to their performance during hypoglycemia compared with normoglycemia or hyperglycemia, in four reports since 2000, and performance in the hypoglycemic range has lagged. The most extensive of these comparisons was presented in 2003 by Chen et al. (12), who measured the precision and accuracy of four widely used meters that command an estimated 90% of the U.S. market. They divided the range of blood glucose readings into hypoglycemic (<70 mg/dl), normoglycemic (84–150 mg/dl), and hyperglycemic (250–400 mg/dl) ranges. The precision and accuracy were lower, in general, for the hypoglycemic readings than for the euglycemic and hyperglycemic readings. The authors concluded that separate accuracy and precision goals should be defined for hypoglycemic, normoglycemic, and hyperglycemic ranges.
Girouard et al. (13) reported in 2000 that in neonates, for blood glucose levels in the 20–72 mg/dl range, the accuracy was slightly lower than that for blood glucose levels >72 mg/dl. Solnica et al. (14) reported in 2001 that for a random assortment of blood glucose levels in a group of subjects with diabetes, the accuracy of a blood glucose monitor was lower for blood glucose levels of <80 and >180 mg/dl than for midrange blood glucose levels between 80 and 180 mg/dl. The precision of the readings in the three ranges was slightly better in the hypoglycemic subjects than in the normoglycemic and hyperglycemic subjects. Skeie et al. (15) reported in 2002 that for four of five blood glucose monitors tested, the percentage deviation from a reference method was highest in the hypoglycemic range compared with the normoglycemic and hyperglycemic ranges. Imprecision was calculated from duplicate measurements by defining deviation as the difference between the first measurement and the mean of duplicate laboratory method results.
A previous study (16) of the accuracy of the GW2B was reported in 2003 by the DirecNet collaborative study group. The investigators determined the accuracy of the GW2B over a range of blood glucose excursions, including hypoglycemia, normoglycemia, and hyperglycemia. Although one of the main reasons for wearing a real-time continuous monitor is for alarm notification of unsuspected hypoglycemia, the GW2B demonstrated its lowest performance in the hypoglycemic range. The median relative absolute difference (RAD) here is defined as the absolute difference of the GW2B glucose value minus the reference value, divided by the reference value, and expressed as a percentage. For serum glucose values >70 mg/dl, the median RAD ranged from 13 to 18%, but for serum glucose values ≤70 mg/dl (which are hypoglycemia-type values), the median RAD was 38%.
Hypoglycemia alarms perform poorly when glucose sensor accuracy is poor. A degradation of GW2B sensor accuracy in the hypoglycemic range explains why the optimal set point (110 mg/dl) for hypoglycemia alarms to detect blood glucose values ≤70 mg/dl has been reported to be associated with as low as 86% sensitivity and 84% specificity (9). In a study (17) of children wearing a GlucoWatch Biographer at a diabetes camp, an alarm threshold of 85 mg/dl was programmed into the devices. The campers reported 20 low-glucose alarms with corresponding meter values measured within 20 min. For this population, there were 10 true-positive alarms, 10 false-positive alarms, and no apparent false-negative alarms. This 50% false-positive alarm rate represented a tradeoff providing perfect sensitivity and imperfect specificity. The performance of any alarm is a function of the alarm’s sensor accuracy. If an alarm is not specific for a given degree of sensitivity, then the percentage of alarms that turn out to be false alarms will be high. At this time, all continuous blood glucose monitors on the market are less accurate than virtually all home blood glucose monitors, which means that almost any alarm on a continuous monitor will be less specific than an alarm would be on a home blood glucose monitor.
A previous study (18) of the accuracy of the CGMS was reported in 2003 by the DirecNet collaborative study group. The investigators determined the accuracy of the CGMS over a range of serum glucose excursions, including hypoglycemia, normoglycemia, and hyperglycemia, in a similar fashion as they did with the GW2B. For serum glucose values >70 mg/dl, the median RAD ranged from 14 to 22%, but for serum glucose values ≤70 mg/dl (which are hypoglycemia-type values), the median RAD was 35%. The investigators measured the accuracy performance of both first- and second-generation CGMS sensors. (Subsequent to the study, the manufacturer of the CGMS discontinued their first-generation sensors, introduced only second-generation sensors to their system, improved their algorithms , and changed the name of their system from CGMS to CGMS Gold.) The investigators also compared their accuracy figures with the ISO performance criteria for glucose monitoring systems (i.e., for reference glucose values ≤75 mg/dl, CGMS values within 15 mg/dl, and for reference values >75 mg/dl, CGMS values within ±0%). For serum glucose values ≤70 mg/dl, only 41% of the first-generation and 48% of the second-generation sensors met ISO criteria. Performance was better in the normoglycemic and hyperglycemic ranges. For serum glucose values >70 mg/dl, CGMS technology met ISO criteria in 45–60% (first generation) and 60–81% (second generation) of sensors. In the future, the CGMS could be equipped with an alarm to detect real-time hypoglycemic events.
In my opinion, both episodic and continuous glucose monitors may detect hypoglycemia less accurately than many clinicians realize. The ability of currently marketed glucose monitoring systems to sensitively and specifically detect hypoglycemia is limited. Patients with diabetes cannot completely depend on any type of continuous monitor with a hypoglycemic alarm to always awaken them from nocturnal hypoglycemia (barring an intolerably high frequency of false alarms). Furthermore, even episodic blood glucose measurements are not always accurate in the hypoglycemic range. Patients must take steps to avoid hypoglycemia by matching food intake to correspond with diabetes medication and exercise, being aware of the earliest symptoms of hypoglycemia, and having access to food (and, in some cases, glucagon) at all times. Glucose testing remains a tool for detecting hypoglycemia, but there is no substitute for common sense.
The two currently marketed continuous glucose monitors (in the U.S.), the GW2B and CGMS, are approved as adjunctive devices (20,21). The information provided by the GW2B must be confirmed with a blood test before action is to be taken, and the information provided by the current-generation CGMS is always reported after the fact, and so no action can be taken based on CGMS results alone. In the future, when new real-time continuous glucose monitors come on the market, their performance in the hypoglycemic range will need to be closely scrutinized, especially if their manufacturers should propose to market them as stand-alone monitors. The current variations in performance of episodic and continuous blood glucose monitors, according to whether a reading is in the low, normal, or high range, call for greater realism in our expectations of this performance. It is now time to present and judge performance of glucose monitors utilizing stratified data according to the magnitude of glycemia.