Accuracy of the 5-Day FreeStyle Navigator Continuous Glucose Monitoring System: Comparison with frequent laboratory reference measurements

Fifty-eight subjects with type 1 diabetes were enrolled in the study. The protocol was approved by an institutional review board. Subjects gave informed consent for their participation in the study. Subjects ranged in age from 18 to 64 years (mean ± SD 40 ± 11 years). Forty-seven subjects were Caucasian. Thirty-six subjects were male. Twenty-one subjects were between 18 and 34 years, 33 were between 35 and 54 years, and 4 were ≥55 years. The length of time since diagnosis was 21.7 ± 11.7 (average ± SD) years, ranging from 0.6 to 43.5 years. BMI was 27.8 ± 4.6 (mean ± SD) with a range of 20.9 to 45.3.

The FreeStyle Navigator Continuous Glucose Monitoring System consists of four components: a miniature electrochemical sensor placed in the subcutaneous adipose tissue, a disposable sensor delivery unit, a radiofrequency transmitter that connects to the sensor, and a hand-held receiver to receive the sensor signal and display continuous glucose values. The electrochemical sensor inserted into the tissue is 5.5 mm long, 600 μm wide, and 250 μm thick. The sensor measures the glucose concentration in the interstitial fluid, which is well correlated with blood glucose (12,13). The sensor is a sterile, single-use disposable element that can be used for 5 days. The glucose measurement is made using the Wired Enzyme method, a modified glucose oxidase enzymatic reaction including an osmium mediator covalently bound to the supporting polymer matrix (14).

The FreeStyle Navigator receiver contains the signal processing algorithms, the user interface, and a display screen for the glucose data. The glucose data on the receiver are updated once a minute and also include a trend arrow to indicate the direction and rate of change averaged over the preceding 15 min. The trend arrows give rates of change of glucose in increments of 1 mg · dl⁻¹ · min⁻¹ from −2 to +2 mg · dl⁻¹ · min⁻¹. The user interface of the receiver allows the threshold alarms to be set at different glucose levels. The projected alarms alert the user when an extrapolated glucose value is predicted to cross the hypo- or hyperglycemia threshold. The user can select the sensitivity of the projected alarms to give a prediction time of 10, 20, or 30 min.

The receiver contains a built-in FreeStyle blood glucose meter for calibration of the continuous glucose sensor as well as for confirmatory blood glucose measurements. The sensor requires four calibrations over the 5-day wearing period at ∼10, 12, 24, and 72 h after sensor insertion. Blood glucose calibration values are accepted for input between 60 and 300 mg/dl and when the absolute rate of change of glucose, determined by the sensor, is <2 mg · dl⁻¹ · min⁻¹. These constraints on the acceptance of calibration input values are designed to limit the potential adverse effects of the intrinsic physiological lag between interstitial fluid glucose and blood glucose. The system does not display continuous glucose values until the first calibration, 10 h after sensor insertion. Before the first calibration, the rate of change of glucose is determined by the raw sensor current calibrated with an initial in vitro sensor sensitivity obtained on a lot-by-lot basis during manufacturing. A second calibration at 12 h is used to corroborate or correct the sensor sensitivity determined by the first in vivo calibration at 10 h.

Subjects were admitted to a health care facility in either the evening or morning. Two sensors were inserted into each subject by trained health care professionals using disposable sensor delivery units. One sensor was worn on the lateral or posterior upper arm and the other on the abdomen. Calibration of the FreeStyle Navigator system occurred at different times of day as well as both pre- and postprandially. Venous sample measurements were taken every 15 min over 50 h of in-patient admission in two or three separate sessions during the 5-day study. Samples were obtained through intravenous placement of an angiocatheter that was flushed once an hour with a heparinized solution. On separate days of in-patient admission, each subject was given an insulin challenge or a 75-g oral glucose load to obtain data during deliberately induced periods of rapidly falling and rapidly rising blood glucose levels. The insulin challenge doses were individualized to achieve blood glucose levels of 60 mg/dl in each subject. The insulin and glucose challenges ensured that there were sufficient data in both hypoglycemic (620 points or 3.0% <70 mg/dl) and hyperglycemic (3,795 points or 18.6% >240 mg/dl) ranges.

Subjects were assigned to different study schedules to provide an approximately uniform distribution of the venous reference data over the total 5-day duration of the study. Subjects were free to move about and perform daily activities except during the glucose and insulin challenges. Data from the sensor and transmitter were stored in the receiver at 1-min intervals but were not displayed to the subjects or clinic staff. All subjects continued with their previously established diabetes management regimens.

Figure 1 shows a typical profile plot for the 5 days of the study with 1-min data from the arm and abdominal sensors as well as the 15-min venous samples taken over two separate periods during the 5 days. The glucose concentration from the venous samples was measured using a YSI 2300 STAT Plus Glucose & Lactate Analyzer (YSI Life Sciences, Yellow Springs, OH). All venous measurements were made in duplicate from single blood samples using a factor of 1.12 to obtain plasma equivalent values (15).

For all 58 subjects, there were a total of 20,362 paired points with both YSI venous measurements and FreeStyle Navigator system interstitial fluid glucose measurements. Table 1 gives the percentage in each zone of the Clarke error grid (16,17). Figure 2 shows the Clarke error grid plotted with color-coded points giving the density of measurements in each 1 mg/dl square of the grid. The plot shows the density as well as the absolute number of points within the clinically accurate A zone (81.7%). There were 16.7% of paired points in the benign error B zone and 1.7% outside of the A and B zones. Results of the consensus error grid are also included in Table 1. The consensus error grid zones have similar clinical significance, but the zone demarcations were defined to eliminate the physical contiguity of the A and D zones in the lower left of the Clarke error grid (18).

On the Clarke error grid, 1.6% of points were in the D zone; 95% of these points were in the left quadrant. On the consensus error grid, the percentage of points in the significant medical risk D zone was reduced to 0.03%. The performance of the FreeStyle Navigator system was also assessed using the mean and median ARD values between sensor interstitial glucose measurements and venous sample measurements. The mean and median ARD values were 12.8 and 9.3%, respectively. The performance of the system on the 5th day was similar to that on the 1st and 2nd days. Table 1 contains the error grid statistics and mean and median ARD values over the full 5 days, by day of wear, by sensor location, by rate of change, and by glucose range. Additional analysis was done to compare the accuracy nocturnally and diurnally. The percentage of points in the Clarke error grid A zone was 87.2% at night and 80.6% during the day.

Sensor accuracy, as measured against venous sample measurements, was analyzed for different rates of change of glucose. The accuracy was highest when the absolute rate of change was <1 mg · dl⁻¹ · min⁻¹ (as measured by the sensor). The sensor accuracy, as measured by the percentage in the Clarke error grid A zone and by the mean and median ARD values, decreased as the rate of change of glucose increased.

The sensor accuracy was highest during periods of euglycemia (70–180 mg/dl) and hyperglycemia (>180 mg/dl) and lowest during hypoglycemia (<70 mg/dl). The mean and median ARD values were 11.7 and 8.9%, respectively, when blood glucose was between 140 and 180 mg/dl; 15.9 and 11.7%, respectively, when blood glucose was between 70 and 140 mg/dl; and 35.7 and 26.4%, respectively, when blood glucose was <70 mg/dl. The international standard for in vitro blood glucose measurement gives the relative difference in units of milligrams per deciliter rather than as a percentage for blood glucose <75 mg/dl (19). Accordingly, Table 1 gives the mean and median absolute differences in milligrams per deciliter for blood glucose <70 mg/dl (19.8 and 15.4 mg/dl, respectively).

Data were postprocessed to assess the accuracy of threshold and projected alarms using the 30-min prediction time for the projected alarm. In 173 instances, the YSI analyzer read <70 mg/dl for 15 min or longer. Hypoglycemia was detected either by the threshold or projected alarms in 79.8% of these instances within ±30 min of the YSI measurements. In an additional 16.2% of instances, the FreeStyle Navigator sensor glucose value was <85 mg/dl, i.e., accurate within the 15-mg/dl limit according to the international standard (19). The FreeStyle Navigator system failed to either provide an alarm or detect hypoglycemia in 4.0% of cases. An alarm was considered false if the associated YSI measurement was >85 mg/dl (i.e., 15 mg/dl high). The false alarm rate for the hypoglycemic threshold alarm was 7.2%. Raising the threshold alarm setting for hypoglycemia to 80 mg/dl resulted in improved detection of hypoglycemic events of <70 mg/dl to 92.5% total, but at the expense of an increase in the false alarm rate to 27.8%.

In 404 instances, the YSI analyzer read >240 mg/dl for 15 min or longer. Hyperglycemia was detected either by the threshold or projected alarms in 79.2% of these instances. In an additional 20.5% of instances, the FreeStyle Navigator sensor glucose value was >192 mg/dl, i.e., accurate within the 20% limit according to the international standard. The FreeStyle Navigator system failed to either alarm or detect hyperglycemia in 0.25% of cases. The false alarm rate for the hyperglycemic threshold alarm was 1.0%, where an alarm was considered false if the associated YSI measurement was <192 mg/dl (i.e., 20% low).

An approximate temporal relationship between the measurements of the FreeStyle Navigator system and the venous reference measurements was determined by applying a time shift to minimize the mean ARD for each subject. This analysis showed an optimal time shift of 12.6 min, consistent with previously published studies using multicompartment diffusion models (13,20,21).

The accuracy of arm and abdominal sensors is shown in Table 1. Data in the table suggest that the performances of sensors in the arm and abdomen were comparable over all glucose ranges and over time. There were small variations in accuracy between arm and abdomen by day, but these were neither clinically nor statistically significant. The precision of matched FreeStyle Navigator measurements on arm and abdomen had a coefficient of variation of 10% (n = 312,953). The trend arrows were in exact agreement for sensors worn on the arm and abdomen 83.6% of the time, within 1 mg · dl⁻¹ · min⁻¹ of one another an additional 14.9% of the time, and discrepant by ≥2 mg · dl⁻¹ · min⁻¹ 1.5% of the time. The alarm performances of the arm and abdominal sensors were equivalent for hypoglycemia (<70 mg/dl) and hyperglycemia (>240 mg/dl) at a 95% confidence level.

Performance of the sensor did not differ as a function of age, sex, ethnicity, or years since diagnosis of diabetes. However, there were differences in the Clarke error grid statistics, depending on the subject's BMI. Subjects with BMI <25.0 kg/m² had 78.8% of measurements in the Clarke error grid A zone (n = 4,844). Subjects with BMI between 25.0 and 30.0 kg/m² had 82.2% of measurements in the Clarke error grid A zone (n = 7,855). Subjects with BMI >30.0 kg/m² had 84.4% of measurements in the Clarke error grid A zone (n = 3,928). The effect of BMI on sensor accuracy was similar for both arm and abdominal sensor placement.

The evaluation of the FreeStyle Navigator system included periods of deliberately induced rapidly rising and rapidly falling blood glucose, i.e., in response to glucose and insulin challenges. Table 1 gives the error grid statistics as well as the mean and median ARD values as a function of the rate of change. Consistent with previously published results, Table 1 shows that 3.1% of the data were associated with a rate of change <−2 mg · dl⁻¹ · min⁻¹, 8.8% were between −2 and −1 mg · dl⁻¹ · min⁻¹, 74.7% were between −1 and 1 mg · dl⁻¹ · min⁻¹, 10.0% were between 1 and 2 mg · dl⁻¹ · min⁻¹, and 3.5% were >2 mg · dl⁻¹ · min⁻¹ (22,23). Whereas 81.7% of paired points were in the Clarke error grid A zone, 84.9% were in the A zone when the absolute rate of change was <1 mg · dl⁻¹ · min⁻¹. Similarly, the mean and median ARDs at these times were 11.7 and 8.5%, respectively.

The percentage of measurements in the Clarke A zone by day of wear ranged from 79.4 to 84.0% with an average of 81.8 ± 1.8% (mean ± SD). The highest percentage of measurements in the Clarke A zone (84.0%) occurred on the 4th day. This finding is most likely due to the absence of glucose challenges on that day of the study. As a result, only 21.2% of the total measurements on the 4th day had a rate of change of >1 mg · dl⁻¹ · min⁻¹ compared with 26.3% on the 2nd day of the study.

There was an apparent increase in accuracy at night compared with the day (87.2 and 80.6% in the Clarke A zone, respectively). This too was likely because of lower average rates of change of glucose at night than during the day, when all of the subjects’ meals were provided as well as when all of the glucose and insulin challenges were conducted. In contrast with previous reports in the literature, there was no evidence of any sustained erroneous hypoglycemic readings at night (24–26).

The introduction of a 10-h delay before the initial calibration is one of the major factors responsible for the difference in accuracy results reported here compared with previous studies using the FreeStyle Navigator system, i.e., 81.7% in the Clarke error grid A zone vs. 68% previously (14). The 10-h delay before the first calibration was instituted to allow time for the tissue around the sensor to achieve equilibrium after insertion. Variable delays in the time required for inserted sensors to achieve stable performance have been observed in many other systems (27–29). In contrast to other continuous glucose monitoring systems that are less accurate on the 1st day than the 2nd (10–11), the 10-h delay in the FreeStyle Navigator system resulted in similar accuracy on the first 2 days as shown in Table 1.

The reduced accuracy in hypoglycemia compared with euglycemia and hyperglycemia is probably multifactorial in origin. The FreeStyle Navigator device was calibrated using capillary blood glucose, whereas the reference measurements were made using venous blood glucose. Capillary glucose is comparable to venous glucose preprandially but can be as much as 20–70 mg/dl higher postprandially (30). The calibration error associated with the capillary venous differential will have a greater relative effect in the hypoglycemic range than in the euglycemic or hyperglycemic ranges.

In addition, the above-mentioned rate of change effect on accuracy was exacerbated because of the greater percentage of high rates of change during hypoglycemia. Ten percent of all data <70 mg/dl was collected when glucose was decreasing more rapidly than 2 mg · dl⁻¹ · min⁻¹ compared with only 3.1% overall. These data had a median absolute difference of 34.9 mg/dl. Similarly, 19% of all data <70 mg/dl were collected with glucose changing between −2 and −1 mg · dl⁻¹ · min⁻¹ compared with 8.8% overall. These data had a median absolute difference of 21.2 mg/dl. However, when the rate of change was between −1 and 1 mg · dl⁻¹ · min⁻¹, the median absolute difference in hypoglycemia was 12.7 mg/dl. Among all hypoglycemic values (<70 mg/dl), 54.5% fell in the Clarke A zone; however, when the rate of change was between −1 and 1 mg · dl⁻¹ · min⁻¹, 62.0% of points were in the A zone.

The accuracy of the FreeStyle Navigator system was slightly greater in subjects with higher BMI than with lower BMI. Tissue glucose nadirs in muscle have been reported to be delayed in time and reduced in magnitude relative to glucose in adipose tissue and blood, especially during insulin-induced hypoglycemia (31,32). In subjects with lower BMI values, the reduced thickness of the subcutaneous adipose tissue layer may result in closer sensor proximity to the underlying muscle tissue. Differences in adipose tissue blood flow observed in subjects with different BMI values may also contribute to the apparent effect of BMI on accuracy (33).

The effect of the physiological lag was the smallest and the accuracy of the device the highest when the rate of change of glucose was small. When the absolute rate of change of glucose was <1 mg · dl⁻¹ · min⁻¹ (74.7% of the time), the FreeStyle Navigator system had a median ARD of 8.5% compared with 5.0% with the latest episodic blood glucose meters (34). Unlike measurements made with blood glucose meters, the trend arrows on the FreeStyle Navigator receiver provide patients with information about the direction and rate of change of their physiological glucose. During high rates of change of glucose, patients may be able to incorporate trend arrow information to make more effective treatment decisions than those based solely on glucose values alone.

Whereas future generations of continuous glucose monitors are likely to improve further, the level of performance reported above (e.g., 81.7% in the Clarke A zone and 9.3% median ARD) may provide significant clinical benefits to patients with diabetes. Retrospective analysis found that the FreeStyle Navigator system was able to provide an alarm or accurately detect hypoglycemia (<70 mg/dl) in 96% and hyperglycemia (>240 mg/dl) in 99.7% of all instances measured by the venous samples. Some patients may elect to use the higher threshold alarm setting at night to further increase detection of hypoglycemia even at the expense of increased false alarms. Future clinical studies will determine whether the combination of the overall accuracy, the alarm performance and the rate of change trend arrows described above result in improved ability of patients to achieve euglycemia.

The authors thank Heather Pryor, Walter Amor, and Hope Buell from Abbott Diabetes Care for analysis of the data.