Which Threshold to Detect Hypoglycemia?: Value of receiver-operator curve analysis to find a compromise between sensitivity and specificity

The Diabetes Control and Complications Trial (DCCT) showed that the improvement of glycemic control by intensive insulin therapy resulted in a decrease in the risk of late diabetic complications. However, this effect was associated with an increase (by a factor of 3) in the risk of severe hypoglycemia (1). This explains the interest in the development of a hypoglycemic alarm that uses continuous glucose monitoring. Several approaches to this are possible.

Noninvasive methods have the benefit of measuring glucose without breaking the cutaneous barrier. Generally, these methods are based on the principle of the analysis of the absorption of the light in a zone of the spectrum close to the infrared, or of the diffraction of the light in the skin. These two parameters are indeed influenced by the level of glycemia, but sensitivity and specificity of these methods are far from providing sufficiently useful miniaturized systems that can be used in clinical practice (2). In invasive methods, the system of measurement (or a part of the system of measurement) is placed inside the body. The system can be totally implantable, requiring that both the sensor and the electronic control system are miniaturized, which is currently possible. In addition, the system must work for a time period sufficiently long to avoid the need for frequent implantations. The sensitive part of the system (the sensor) may be placed directly into a blood vessel (3) or in subcutaneous tissue (4). Nevertheless, for obvious reasons, most of the invasive systems that have been developed so far use transcutaneous access.

Two basic approaches have been attempted. In the first approach, the glucose sensor has the shape of a needle that is implanted under the skin and is changed by the patient approximately every 2 to 4 days. The sensor is connected with a cable or through telemetry to an electronic system that processes the data and displays the glucose result or triggers an alarm. This approach, imagined 20 years ago by Shichiri et al. (5), was pursued by several teams (6,7,8). A system, developed by MiniMed (7) is available in the form of a signal recording system. This system can therefore capture the glycemic profile of patients during the implantation period (3 days), but it cannot currently be used in real time as an alarm for hypoglycemia. In the second basic approach, a microdialysis tube is implanted under the skin, also in a repeated manner. A pump circulates a liquid through this tube, into which glucose diffuses. The liquid is collected in a device that includes a glucose sensor. The external device can be miniaturized to contain both the pump and glucose sensor (9–10).

A minimally invasive method consists of measuring the glucose in a liquid extracted from the skin either by suction or by iontophoresis. In iontophoresis, when electrical potential is applied to the surface of the skin, ions migrate outward. This ion migration drives water, which contains glucose in micromolar concentration. The glucose collected in this manner can then be measured by a sensor. This system was developed by Cygnus (11) under the name of the GlucoWatch biographer. It provides an estimation of the concentration of glucose in the subcutaneous tissue every 20 min for up to 12 h after a 3-h equilibration period, using a single blood glucose measurement for calibration. The reading lags behind the corresponding blood glucose value by ∼17 min. The system is capable of detecting events that may produce an erroneous result, such as excessive perspiration. In that case, the measurement is skipped. A slight, transient, cutaneous reaction is sometimes observed (12,13).

In an article published in this issue of Diabetes Care, Pitzer et al. (14) of Cygnus describe the ability of this system to detect hypoglycemic events, defined as blood glucose <3.9 mmol/l (70 mg/dl). They analyze results obtained from several studies with type 1 or type 2 diabetic patients. These studies were carried out in a controlled clinical environment, a simulated home environment, and a home environment. The different studies gave similar results, and those obtained at home are described in detail. The home environment study lasted 5 days, with one biographer use each day. In addition, the patients made one measurement of blood glucose per hour. A total of 420 biographer uses were reported to produce >12,000 data points. Of those biographer measurements, 3,060 were compared with the concomitant measure of the capillary blood glucose concentration determined with a One Touch Profile system, obtained 10–20 min before the reading by biographer. Of the 3,060 paired measurements, 160 (5.2%) determinations by the One Touch Profile system were <3.9 mmol/l. Of these 160 hypoglycemic events, the biographer correctly detected 39 events (sensitivity 24%). The blood glucose concentration was >3.9 mmol/l in 2,900 cases. The biographer mistakenly produced a value <3.9 mmol/l in 36 cases, which gives a specificity of 1–36/2,900, >99%. Clearly, if a threshold of 3.9 mmol/l is fixed to detect a blood glucose concentration <3.9 mmol/l, the specificity of the answer is excellent, but the sensitivity is too low. If the alarm threshold is increased, the sensitivity is increased at the cost of a lower specificity.

The authors used the technique of the receiver-operator characteristic (ROC) curve (16), which provides a method to find the threshold that gives the best compromise between sensitivity and specificity. In the four studies, this threshold was found to be 5.6 mmol/l, where sensitivity and specificity were 75 and 10%, respectively. These results may represent a major advance in the detection of hypoglycemia, considering that they were much better than that provided by the discontinuous measurement of blood glucose performed twice (sensitivity 14%) or even four times a day (sensitivity 39%) before a meal, i.e., what our patients currently do. It is clear that the threshold of detection of the biographer can be adjusted according to the need. If one wishes a very good sensitivity, the threshold will be increased, in which case the alarm will often be activated for nothing, but the user will not miss a hypoglycemic episode.

This study gives for the first time an evaluation of the ability of a continuous glucose monitoring system to detect hypoglycemic events and demonstrates the interest of the continuous measure of glycemia, even measured every 20 min. It also suggests that, as for any measure, sensitivity and specificity cannot be 100% and that it is necessary to find a compromise. Incidentally, differences between the estimation by the biographer and the simultaneous measurement of the capillary glycemia may not be caused only by an imperfection of the biographer. After all, the commercial systems of the measurement of glycemia have ≥10% error (16). Furthermore, the biographer estimates glucose concentration in interstitial tissue, which can be different from the concentration in blood, especially during rapid changes, when there is variation in the time lag between the two measurements (17,18).

The analysis by the ROC curve is very attractive because it provides a method to find the optimal threshold, which gives a slope of 1 on plots of sensitivity versus specificity. In addition, it shows how variation of the threshold can affect the respective values of the two parameters. This analysis also suggests that despite discrepancies between the real glycemia and the estimation by the system mentioned above, detection of hypoglycemia by this type of system was relatively independent of the conditions of the analysis. Interestingly, the same ROC curve was obtained for the various conditions of study, the clinical research set-up, and the use at home. The area under the ROC curve was 0.91 (an ideal method would yield a value of 1), which means that this diagnosis test is useful (19). However, it is necessary to remain careful in the extrapolation of these results, which pooled the data of several hundred biographer uses into the sensitivity and the specificity of a given application of the device (the precision of which can be jeopardized by an error in the calibration). It remains to be determined whether the detection of 75% of hypoglycemic episodes will be sufficient to reassure patients and whether 10% of readings false-positive will add to their concern about hypoglycemia. Long-term local tolerance must be also considered. Clearly, the impact of this new technology must be estimated in terms of the prevention of severe hypoglycemia, metabolic control, and the quality of life.