1018-P: Evaluating Continuous Glucose Monitoring Documents in Electronic Health Records—A Comparative Study of Algorithmic Classification vs. Manual Review

Introduction & Objective: Clinical use of continuous glucose monitoring (CGM) is increasing collection and storage of CGM-related documents in electronic health records (EHR); however, the standardization of CGM storage is lacking. We aimed to evaluate sensitivity and specificity of CGM Ambulatory Glucose Profile (AGP) classification criteria. This work is crucial to automatically obtain AGP for real-world CGM research.

Methods: We retrieved 12,415 CGM-related documents from the EHR at NYU Langone Health 2012-2022. We randomly chose2244 (18.1%) documents to evaluate AGP classification criteria. Our document classification algorithm: 1) separated multiple-page document into a single-page image; 2) rotated all pages into an upright orientation; 3) applied optical character recognition to the image; 4) tested for the presence of particular keywords in the text. Two experts in using CGM for research and clinical practice conducted an independent manual review of 62 (2.8%) reports. We calculated sensitivity (correct classification of CGM AGP report) and specificity (correct classification of non-CGM report) by comparing the classification algorithm against manual review.

Results: Among 2244 documents, 1040 (46.5%) were classified as CGM AGP reports (43.3% FreeStyle Libre and 56.7% Dexcom), 1170 (52.1%) non-CGM reports (e.g., progress notes, CGM request forms, or physician letters), and 34 (1.5%) uncertain documents. The agreement for the evaluation of the documents between two experts were 98.4%. When comparing the classification result between the algorithm and manual review, the sensitivity and specificity were 95.0% and 91.7%.

Conclusion: Nearly half of CGM-related documents were AGP reports, which are useful for clinical practice and diabetes research; however, the remaining half documents are administrator files. Future work needs to standardize the storage of CGM-related documents in the EHR.