We conducted a multicenter evaluation of a novel remote foot-temperature monitoring system to characterize its accuracy for predicting impending diabetic foot ulcers (DFU) in a cohort of patients with diabetes with previously healed DFU.
We enrolled 132 participants with diabetes and prior DFU in this 34-week cohort study to evaluate a remote foot-temperature monitoring system (ClinicalTrials.gov Identifier NCT02647346). The study device was a wireless daily-use thermometric foot mat to assess plantar temperature asymmetries. The primary outcome of interest was development of nonacute plantar DFU, and the primary efficacy analysis was the accuracy of the study device for predicting the occurrence of DFU over several temperature asymmetry thresholds.
Of the 129 participants who contributed evaluable data to the study, a total of 37 (28.7%) presented with 53 DFU (0.62 DFU/participant/year). At an asymmetry of 2.22°C, the standard threshold used in previous studies, the system correctly identified 97% of observed DFU, with an average lead time of 37 days and a false-positive rate of 57%. Increasing the temperature threshold to 3.20°C decreased sensitivity to 70% but similarly reduced the false-positive rate to 32% with approximately the same lead time of 35 days. Approximately 86% of the cohort used the system at least 3 days a week on average over the study.
Given the encouraging study results and the significant burden of DFU, use of this mat may result in significant reductions in morbidity, mortality, and resource utilization.
Introduction
Diabetic foot ulcers (DFU) are a common, limb-threatening, and expensive complication of diabetes (1,2). Of the ∼30 million patients with diabetes in the U.S., 1.7 million suffer one or more DFU annually (3). Conservative estimates of DFU-related direct costs in the U.S. exceed $17 billion (2,4). Those who have recently healed from a DFU episode are especially likely to suffer reulceration (1,5–9). A principal goal of care for these patients is to maintain the integrity of the newly formed epithelium and allow the underlying tissue to complete remodeling.
Unfortunately, caring for the patient in remission following a DFU episode has proven challenging under standard practice. Numerous prospective studies have explored how ulcer-free survival is impacted not only by underlying comorbidities, but also by the duration since the patient most recently healed from a DFU episode (1,5–9). These investigations suggest that between 30 and 40% of patients experience a recurrent DFU in the year after healing. Contrast this with the baseline incidence among all patients with diabetes, which has been measured between 3.6 and 5.8% (10,11). Thus, a better approach focused on predicting DFU to enable targeted preventative therapies during this critical period may significantly improve patient outcomes and reduce DFU-related costs (12).
Skin-temperature monitoring first emerged in the 1970s as a potentially useful tool for identifying patients at risk for ulceration (13,14). For those wounds that are not because of acute injury, the hypothesized mechanism is believed to be repetitive microtrauma leading to localized enzymatic autolysis of tissue and inflammation. The temperature-monitoring approach best supported in the literature (15–18) uses plantar foot temperature asymmetry between a pair of feet to identify patients with elevated risk. This practice is referred to in this study as “asymmetry analysis.”
Asymmetry analysis coupled with early offloading of foot pressure has been shown to reduce DFU incidence by ∼70% across three randomized controlled trials (15–17). An independent systematic review and meta-analysis determined the summary odds ratio (OR) of the three trials to be 3.84 (95% CI 1.50–6.17) (18). The threshold most commonly used for starting preventative therapy has been 2.22°C (4°F) over at least 2 consecutive days, although no studies have been published to derive or optimize this threshold.
Despite evidence from multiple randomized controlled trials and inclusion in three clinical practice guidelines (19–21), temperature monitoring remains uncommon in practice. This may be because of challenges with previous temperature-monitoring technologies, including onerous patient workflow and poor usability.
A telemedicine foot mat has been developed to encourage adoption of daily foot-temperature monitoring in accordance with existing practice guidelines. We therefore undertook this investigation with the following objectives: 1) evaluate the effectiveness of the foot mat as an early predictor of recurrent plantar DFU in high-risk patients, 2) determine participant adherence to daily use of the mat over time, and 3) understand user perceptions of the mat and ease of use.
Research Design and Methods
We conducted a prospective, multicenter, cohort study to evaluate a novel remote foot-temperature monitoring system (ClinicalTrials.gov Identifier NCT02647346). The study device is a wireless thermometric foot mat. We enrolled 132 diabetic participants, each with history of healed DFU prior to enrollment, across seven outpatient sites in the U.S. representing varied care environments. The study was approved by the New England Institutional Review Board and local review boards for VA Phoenix (Phoenix, AZ), VA Long Beach (Long Beach, CA), VA Miami (Miami, FL), and Greenville Health System (Greenville, SC). The follow-up period for each participant was 34 weeks from the time of enrollment. The primary outcome end point was occurrence of nontraumatic plantar DFU. The primary efficacy analysis was based on the accuracy of the study device for predicting the occurrence of nontraumatic plantar DFU prior to clinical presentation. Secondary outcomes included participant adherence to the daily use of the mat, device-related trips or falls, and device-related injury.
Study Population
Inclusion criteria consisted of a diagnosis of diabetes documented in the medical record (type 1 or 2), age >18 years, and history of a healed prior plantar DFU. Exclusion criteria included an ankle-brachial index documented in the medical record ≤0.5, history of major lower extremity amputation (i.e., above ankle), open plantar wound (including DFU), active Charcot foot disease, end-stage renal disease, active malignancy, immunosuppressive disease, cognitive deficit preventing adequate participation, or any other issue that, at the discretion of the investigator, rendered the participant ineligible for participation.
Data Collection
Enrolled participants underwent a detailed history and physical at the beginning of participation. Participants received the study device, were trained in its proper use and function, and completed a baseline foot scan. Participants were instructed to place the device in a convenient location within the home and to stand on it for 20 s daily. Participants returned devices to the enrolling site upon completion of or withdrawal from the study, at which time each participant completed a foot exam, a final scan with the study device, and a brief usability questionnaire.
All participants received standard medical and preventative diabetic foot care at the discretion of the managing physician, including appropriate footwear, instructions to continue daily foot inspections, and instructions to contact their clinician and principal investigator upon discovering any lesion. Participants developing DFU were not withdrawn from the study, allowing treatment of multiple DFU to a single participant as independent events. Participants developing a plantar DFU during participation were instructed to discontinue use of the study device for the duration of the episode. If the DFU healed during participation in the study, he or she was subsequently encouraged to resume using the study device.
All scans collected were timestamped, allowing for an assessment of which days a scan was successfully completed by each participant. For purposes of determining longitudinal adherence, we treated multiple scans collected from a participant in a given day as a single use, and we excluded days during which a patient had a contraindication to using the mat (e.g., for open plantar wound).
Participants were contacted by a study coordinator after 4 consecutive days of not using the mat. A maximum of eight calls were made to any given participant during the study. Participants who did not use the mat for >28 consecutive days were deemed lost to follow-up. Adherence was evaluated using both a per-protocol and an intention-to-treat (ITT) approach. The former characterized adherence for each subject until either the day participation concluded or the participant became lost to follow-up. The ITT analysis characterized adherence data across the entire study regardless of whether or not a participant became lost to follow-up. If a participant withdrew consent prior to study completion, we included all data collected from the participant from enrollment through the day consent was withdrawn.
Study Device
The study device is an in-home telemedicine system designed to enable remote temperature asymmetry monitoring and analysis (Remote Temperature Monitoring System; Podimetrics, Inc., Somerville, MA). The system includes a daily-use, wireless floor mat with an array of temperature sensors under a water-resistant cover called the Podimetrics Mat (Fig. 1). It is designed to be used without configuration or setup by the patient. It has a low profile with tapered edges to prevent tripping, a large footprint (∼30 by 43 cm) to allow a comfortable stance while in use, and a conformable foam base to allow intimate contact between the bottoms of the feet and the sensors. It is designed to accommodate the foot length, stance width, and weight of >99% of the American population.
The device remains in standby until the patient is ready to start a scan, which is accomplished by stepping on the mat and remaining stationary for ∼20 s. During this time, the device records a temperature scan, or thermogram, of the feet from data from ∼2,000 thermistor sensors. The thermogram has an accuracy of ±0.6°C (±1°F) and a precision of 0.1°C, and the device is accurate over a temperature range of 15 to 40°C. After notifying the patient that the scan is complete, the mat transmits the scan data wirelessly and securely to Health Insurance Portability and Accountability Act of 1996–compliant servers managed by the manufacturer. The data are saved and processed, and the foot temperature asymmetry is automatically calculated based on the thermogram.
The study device is legally marketed in the U.S. as a class I medical device (product code OIZ Daily Assist Device; 510[k] designation K150557) and has been cleared by the U.S. Food and Drug Administration for its intended use of “periodic evaluation of the temperature over the soles of the feet for signs of inflammation.”
Analysis Plan
We compared two subcohorts: those who developed at least one DFU during the study and those who remained ulcer-free throughout participation. To make between-group comparisons over continuous variables, we used the independent t test with Welch correction for unequal population variances. For comparisons of proportions between groups, we used the Fisher exact test to evaluate independence. For all comparisons, we set α = 0.05 as the threshold for significance. Given these direct comparisons, we completed a multiple logistic regression including all variables that were significant at the α = 0.05 level to minimize the influence of multicollinearity, which we anticipated to be relevant among several covariate subsets.
Effect sizes for continuous variables were reported using Cohen’s d statistic, and ORs were used for proportion effect sizes. These were categorized as “small,” “medium,” and “large” per the conventions of Cohen (22,23). Specifically, for comparison of continuous variables, Cohen’s d values of 0.2, 0.5, and 0.8 were considered small, medium, and large effect sizes, respectively. For comparison of proportions, ORs of 1.45, 2.5, and 4.3 were considered small, medium, and large effect sizes, respectively.
To evaluate classification accuracy, we constructed a receiver operator characteristic (ROC) curve that defined the sensitivity and specificity of the prediction as a function of temperature asymmetry threshold. False-positive and false-negative rates were calculated over 2-month samples of participant data. Reporting these statistics over a 2-month interval allows for a more clinically meaningful and consistent interpretation of the results commensurate with a hypothesized duration between office visits for a high-risk patient. Another benefit of this approach is that it implicitly weights the outcomes for each participant by the quantity of data collected for that participant, naturally handling participants with censored data because of developing a clinical contraindication. This approach also allows for unambiguous treatment of participants who suffered multiple DFU events during the study, whereas the more traditional approach of aggregating data on a per-participant basis would underreport the potential impact of the study device for those patients at highest risk. Finally, using 2-month intervals for reporting better ensures causality between the thermometric signal measured and the development of any subsequent DFU given the long duration of follow-up.
For the purposes of the ROC analysis, we considered true-positive cases those in which a given temperature asymmetry threshold was exceeded in any two consecutive scans prior to the participant developing a DFU in the same 2-month interval. False-positive cases were those in which a given threshold was exceeded but the participant did not develop a subsequent DFU during the 2 months. Random 2-month intervals of participant data were sampled to assess the false-positive and true-positive rates. We obtained each 2-month interval by randomly selecting a participant and then randomly selecting a start date from which to index a 2-month interval within period of participation. Given the large number of potential 2-month intervals, we conducted a sensitivity study to determine how many samples were necessary to estimate the false-positive and true-positive rates with confidence. We increased the number of samples until the ROC area under the curve converged to two significant digits. Convergence was initially achieved at 10,000 samples.
To justify pooling data across the seven sites, we compared the observed DFU incidence (on both a per-patient-year and a per-patient basis) among all site-to-site permutations. We also compared site-by-site incidence against the pooled estimate. None of the comparisons yielded statistically significant differences in incidence at α = 0.05. From this, we concluded any center heterogeneity was negligible.
Results
Table 1 summarizes the demographic characteristics of the cohort across the seven enrolling sites with evaluable participants (N = 129). Although 132 participants were enrolled, 3 participants from an eighth site were subsequently removed from the study because it was closed early for unevaluable data. A total of 37 participants (28.7%) presented with 53 DFU (0.62 DFU/participant/year) during the study.
Device Accuracy
Two subcohorts were compared in this high-risk population: those who developed at least one new or recurrent DFU during the study and those who did not. Interestingly, none of the demographics or characteristics were found to be a statistically significant predictor of the development of DFU in this cohort, although participant weight, BMI, and insulin use were found to be correlated with DFU occurrence at nearly significant levels. Furthermore, we found all effect sizes for participant demographics to be small. The largest of these were weight (Cohen’s d = 0.37; 95% CI 0.02–0.75; P = 0.06), insulin dependence (OR 2.2; 95% CI 1.04–5.0; P = 0.07), and BMI (Cohen’s d = 0.32; 95% CI 0.07–0.70; P = 0.09).
In contrast to these results, the maximum temperature asymmetry of scans in the 2 months preceding clinical presentation of a DFU was strongly differentiated from the maximum asymmetry of scans from randomly sampled 2-month intervals from participants who did not develop a DFU (Cohen’s d = 0.79; 95% CI 0.76–0.81; P < 0.01). This nearly constitutes a “large” effect size.
Table 2 presents the predictive accuracy of the study device over a range of temperature asymmetry thresholds that span sensitivity and specificity ranges that we believe may find use in clinical practice. At 2.22°C, the system correctly identified 97% of observed DFU, with an average lead time of 37 days with a false-positive rate of 57%. Extrapolating over a year by assuming the true-positive and false-positive rates are constant and equal to those observed during the 34-week trial, we would expect ∼3.1 notifications per participant per year. Although only four discrete temperature thresholds are presented, the values in Table 2 can be interpolated to estimate performance at different thresholds among those given.
Fig. 2 compares two typical and comparable participants: one who did not develop a recurrent DFU during the study (subject I, left panel), and one who did (subject II, right panel). The top row of Fig. 2 presents the longitudinal temperature asymmetries over time for both participants. The second row presents two thermograms from each participant collated by caption (A, B, C, and D) to the asymmetry timeline history.
Subject I is a 61-year-old male participant with a history of DFU at the left hallux (closed 40 weeks prior to enrollment) and the right hallux status postamputation (healed 42 weeks prior to enrollment). At no time during study participation did he exceed any of the temperature asymmetry thresholds in Table 2, and the participant remained ulcer free.
Subject II is a 59-year-old female participant with a history of DFU at her right hallux and right 5th metatarsal head with no history of surgical intervention. Her most recent DFU (right 5th metatarsal head) healed 11 weeks prior to enrollment. Temperature asymmetry exceeded 2.22°C at multiple time periods during participation, and her right fifth metatarsal head DFU subsequently recurred by week 10. On presentation, the wound was evaluated to be University of Texas Diabetic Wound Classification 1A (superficial without ischemia or infection). The inflammation associated with her emergent DFU is clearly visible in Fig. 2, panel D.
Participant Disposition and Adherence
Of the 129 participants who had evaluable data, 14 (10.9%) withdrew consent prior to completion, 3 (2.3%) died, and 24 (18.6%) were lost to follow-up. The most common reasons for early withdrawal were occurrence of a significant adverse event not related to the device or participation in the study (3.9%), “personal reasons” (2.3%), and poor wireless signal strength (1.6%). In contrast, only one participant withdrew consent because of difficulty using the mat (0.8%).
Fig. 3 characterizes participant adherence to the daily use of the device using both an ITT and a per-protocol approach. The top row (Fig. 3A and B) represents adherence data from the study cohort using an ITT approach; the bottom row (Fig. 3C and D) represents adherence data from the study cohort using the per-protocol analysis. The left columns (Fig. 3A and C) display histograms stratifying the participant population by the average number of uses per week over the entire study duration. The right columns (Fig. 3B and D) display the average uses per participant per week of the study cohort over time.
Per ITT, 86% of the cohort used the system 3 days/week or more. Mean adherence over the study period was 5.0 days/week, with modest decay over time because of participants becoming lost to follow-up. Per protocol, the percentage of participants using the system >3 days/week improves to 94.6%, with mean adherence increasing to 5.5 days/week.
Nearly all participants (98.4%) were able to set up and use the device at home without assistance. When asked how easy the study device was to use on a scale of 1 (very hard) to 4 (easy), 51 of 58 respondents (88%) reported an ease of use of 4. No device-related adverse events were reported during the study. The most commonly noted adverse event was DFU recurrence.
Conclusions
We completed a multicenter evaluation of a novel remote temperature-monitoring system to characterize its predictive accuracy and usability. Our results suggest that plantar temperature asymmetry was highly predictive of impending DFU. In addition, we examined different temperature asymmetry thresholds and their impact on prediction sensitivity and specificity, which represents a novel and previously uncharacterized aspect of temperature monitoring of the diabetic foot.
Using an asymmetry threshold of 2.22°C, the standard threshold used in previous studies (15–17), the mat was able to detect 97% of nontraumatic DFU ∼5 weeks before they presented to the participant and/or clinician. These data are consistent with and extend the work of previous researchers. Additionally, the data support clinical practice guidelines that emphasize incorporating daily thermometry into standard preventative care (15–17,19–21,24).
The proportion of participants who developed a DFU during this investigation is higher than previous studies with similar enrollment criteria (15,16). However, it is difficult to make a direct comparison because prior studies did not characterize the duration between when a participant healed from their most recent DFU episode and when they were enrolled. This is known to be a significant confounder (1,5–9). We note a median duration of 2.9 months from previous closure among our participants, which potentially explains the high observed incidence in part.
Despite the common impression that in-home foot-temperature monitoring is unrealistic for this population, daily adherence was encouraging, with 86% of the cohort averaging at least three uses per week per an ITT analysis. Although this is the first study to objectively examine longitudinal adherence of which we are aware, it has been previously demonstrated that patients with diabetes are poorly adherent to therapeutic interventions, including prescribed pressure-offloading strategies (25,26). The strong adherence could be because of the automation and connectivity designed into the study device, which enables continuous surveillance of adherence and re-engagement when necessary, and the simplicity of the mat form factor, which is supported by 88% of respondents reporting it to be “easy” to use. Despite this success, it is important to note that losses to follow-up were observed, suggesting the system may not be uniformly adopted by all patients.
High adherence may enable reductions in DFU incidence beyond the 70% previously demonstrated (15–17). Of the three randomized controlled trials that evaluated temperature-guided avoidance therapy, one paper (16) characterized the impact of poor adherence on prevention. They noted four of the five participants that ulcerated in the treatment group were nonadherent to the prescribed monitoring regimen. It is therefore conceivable that improved adherence may result in larger reductions in incidence.
In addition, one potential benefit of the system suggested by these data but not yet investigated is the utility of the study device for evaluating the effects of offloading interventions (e.g., new custom shoes, which should, in theory, reduce pressure at sites of previous ulceration). Further, individual thresholds can be tailored or several used concurrently to customize both the balance between sensitivity and the number of alerts produced as well as the resulting clinical response. Similarly, one potential use of the telemedicine foot mat that has yet to be explored is categorizing various types of temperature patterns and testing the impact of different therapeutic approaches to each. It is conceivable that different pathologies present with different thermometric phenotypes, allowing the thermal data to better inform appropriate care. Future studies should be explored to understand this relationship and impact on clinical practice.
This study has three important limitations that should be considered. First, because the study was noninterventional, we did not characterize potential reduction of DFU incidence and related costs. The study design was chosen to characterize the accuracy of the study device, and notifying clinicians of incipient DFU would have confounded this outcome of interest. Because potential impact of temperature-guided avoidance therapy is already well established, this study was needed to address a gap in existing understanding: namely, what is the tradeoff between sensitivity and specificity as a function of temperature asymmetry threshold? Future trials on this technology should focus on measuring how the device affects both clinical and financial outcomes, as well as how these effects may be optimized by adjusting system sensitivity and specificity.
Second, it should be noted that the conclusions drawn in this study must be limited to the population studied (i.e., those with a prior history of DFU). Though all patients with diabetes are at risk for developing DFU, future studies are needed to clarify classification accuracy and utility in lower-risk populations.
Lastly, we reported sensitivity and specificity over an interval of 60 days. This should be considered when evaluating the accuracy statistics reported. For example, if a DFU presented 65 days after the threshold was first exceeded, the sequence could potentially be counted as both a false-positive (notification not followed by an DFU within 60 days) and a false-negative (DFU not preceded by a notification within 60 days), even though the alert may have correctly identified the physiology of the developing DFU.
Despite inclusion in several recent clinical practice guidelines (19–21), routine home monitoring of plantar temperatures is infrequent in actual practice. This is likely the result of limitations with current foot-temperature monitoring technologies, which present a number of challenges: an onerous patient workflow, a requirement that the patient maintain meticulous notes, and a reliance on the patient to identify alerts and trigger the intervention. In addition, without a connected solution that can be monitored remotely, providers are unable to offer support when adherence decreases or ensure that every alert will result in a successful communication with the patient. In this study, we evaluated a technology that may potentially address these shortcomings, making daily home monitoring of foot temperatures more practical.
This study was successful in its three main goals: evaluating the effectiveness of the mat as an early detector of plantar DFU, determining participant adherence to using the mat over time, and understanding participant perceptions of possible benefits and ease of use. Notably, the mat detected as many as 97% of developing foot ulcers on average ∼5 weeks before they presented clinically, resulting in a modest increase in patient interactions with the health care system with 3.1 notifications annually at the most sensitive setting presented in this study. Furthermore, strong acceptance of this technology by the participants was suggested, as 86% of the population used the system at least 3 days a week on average over the entire duration of the study. Given the significant burden of DFU, use of this mat may result in significant reductions in morbidity, mortality, and resource utilization.
Clinical trial reg. no. NCT02647346, clinicaltrials.gov.
Article Information
Funding. Podimetrics, Inc. funded this study as its sole sponsor, including providing the study devices and financial support to the clinical sites for study operations.
Duality of Interest. J.D.B., B.J.P., and D.R.L. are employees of Podimetrics, Inc., a private company that designed and manufactured the study device and provided financial support to clinical sites for study operations as the sole sponsor of the study.
They have contributed to data analysis, drafting, and interpretation of results. However, they were not involved in patient recruitment, consenting, or patient care. No other potential conflicts of interest relevant to this article were reported.
Author Contributions. R.G.F. was responsible for study concept and design, clinical supervision, patient care, patient recruitment, acquisition of data, interpretation of data, and preparation of the manuscript. I.L.G. was responsible for clinical supervision, patient care, patient recruitment, acquisition of data, interpretation of data, and preparation of the manuscript. A.M.R. was responsible for clinical supervision, patient care, patient recruitment, acquisition of data, and interpretation of data. S.M.C. was responsible for clinical supervision, patient care, patient recruitment, acquisition of data, and interpretation of data. R.H.F. was responsible for clinical supervision, patient care, patient recruitment, acquisition of data, and interpretation of data. G.M.R. was responsible for clinical supervision, patient care, patient recruitment, acquisition of data, and interpretation of data. J.D.B. was responsible for study design, data analysis, investigator training, data acquisition, interpretation of data, and preparation of manuscript. B.J.P. was responsible for concept and design, data analysis, investigator training, data acquisition, interpretation of data, and preparation of the manuscript. D.R.L. was responsible for data analysis, investigator training, data acquisition, interpretation of data, and preparation of the manuscript. A.N. was responsible for study design and interpretation of data. B.N. was responsible for concept and design, clinical supervision, patient care, patient recruitment, acquisition of data, and interpretation of data. All authors contributed in critically revising the manuscript and have given final approval of the version to be published. R.G.F. is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Prior Presentation. Parts of this study were presented in abstract form at the 76th Scientific Sessions of the American Diabetes Association, New Orleans, LA, 10–14 June 2016.
Appendix
Investigators and Enrolling Centers: R.G.F., Phoenix VA Health Care System, Phoenix, AZ; I.L.G., VA Long Beach Healthcare System, Long Beach, CA; A.M.R., Center for Clinical Research, Castro Valley, CA; S.M.C., Limb Preservation Platform, Fresno, CA; R.H.F., Greenville Health System, Greenville, SC; G.M.R., Miami VA Healthcare System, Miami, FL; and B.N., Banner University Medical Center Tucson, Tucson, AZ.