911-P: Independent Accuracy Verification of a Smartphone App’s Blood Glucose Prediction Algorithm

Smartphone applications have become important health management tools for patients with diabetes. In particular, short-term blood glucose predictions offered by some apps can help reduce the frequency and duration of hypo- and hyperglycemic events, which makes the accuracy of such predictions essential for better glycemic control.

In this paper, we describe the results of an in-silico study designed to measure the predictive accuracy of the algorithm of our smartphone app. In this study, an FDA-approved blood glucose simulator was used by an independent third party to generate 180 days of realistic glycemic data for 300 virtual patients under 3 different study protocols. This data (about 225 million blood glucose points with accompanying food, insulin, and exercise information) was sent consecutively, one temporal point at a time to avoid the possibility of future data leakage, to our predictive module, which generated personalized predictions of blood glucose levels for the next 60 minutes and sent the results back to the third party to be used in accuracy metrics calculation. Several versions of the predictive algorithm were tested, including one using only CGM data, and another that also took into account recent food, insulin, and exercise events.

In the end, for CGM-only models trained on 20,000 prior blood glucose points from the same patient, 93.16% of 30-minute predictions and 79.54% of 60-minute predictions ended in Parkes Zone A (clinically accurate), and 99.71% of 30-minute predictions and 98.82% of 60-minute predictions in Parkes Zones A and B (clinically acceptable). For the models that used additional inputs, Parkes Zone A percentages improved to 98.67% of 30-minute predictions and 92.51% of 60-minute predictions, while Parkes Zones A and B percentages rose to 99.99% of 30-minute predictions and 99.72% of 60-minute predictions. For both models, the high degree of accuracy achieved in the study shows the usability of our approach for real-time glycemic predictions.