We are grateful for the opportunity to respond to the letters (1–3) concerning our recent study (4). Unfortunately, we cannot respond to every comment in the space allotted, so we have opted to focus on general themes. Of note, many of the concerns in the letters are already detailed in our article.
Many of the writers had concerns about our study design (1–3). We conducted an effectiveness study or an evaluation of how well treatment works in a real-world setting. Ours was not a randomized clinical trial (RCT) but a cohort study. This is fundamentally different from an efficacy study, which assesses how well a treatment works in an idealized and highly controlled setting. Efficacy studies are usually RCTs that establish the scientific basis that a treatment works in a predefined population under a detailed prespecified treatment protocol. Effectiveness studies, on the other hand, are arguably more generalizable than efficacy studies because they are larger, take place in diverse settings (e.g., many wound care clinics with varying clinical practices) with diverse patient populations, and are conducted in a real-world setting. The results from these two designs are often different. Of interest, the meta-analyses of hyperbaric oxygen therapy (HBOT) could not combine the RCTs because they were too “heterogeneous”; sometimes even carefully designed and controlled RCTs are not similar (5).
Our study did not use randomization as a method to control for selection bias, a general concern for cohort studies, but effectively used two widely used and theoretically sound statistical approaches: propensity scores (to adjust for measured confounders) and instrumental variables (to account for both measured and unmeasured confounders). Both methods are carefully described in our article and the Supplementary Data. Propensity scores are commonly used to analyze nonrandomized studies; in fact, the original article that proposed this statistical approach, per Google Scholar, has been cited over 9,000 times (6). It is true, as discussed in the letters, that propensity scores only adjust for measured covariates, and it could be possible—as discussed in our article—that the induced covariate balance was not sufficient to control for selection pressures in the setting of HBOT. However, unlike a mentioned concern, the propensity score did nicely balance covariates known to be associated with the failure of a wound to heal (2). Further, we feel that it is highly unlikely that in our setting the propensity adjustment made the bias worse (as posited by Carter et al. [1] and Sherlock [3]) because the propensity score adjustment decreased the estimated effect of HBOT toward 1 (no effect) and not in the other direction (1,3). We also conducted an analysis using propensity score matching and found similar results, so it is very unlikely that wound size was not properly balanced (3). We did evaluate groups separately by wound grade because this variable was not as well balanced as others and were still unable to find a healing effect of HBOT (3). In order to address whether our result was robust to the presence of unmeasured confounding, we conducted a careful sensitivity analysis. Although most sensitivity analyses account for a single unmeasured confounder it is feasible that a combination of confounders taken together in a statistical model may have a similar distribution as the single confounder and would have the same effect. This analysis revealed that an unmeasured factor that would dramatically change our results was unlikely. However, to further address the limitations of our analysis, we used another technique, instrumental variables analysis, which is based on a statistical construct different than propensity scores, to account for both measured and unmeasured confounders. The results were nearly identical between the two approaches.
Writers also opined that number of HBOT sessions was insufficient for about 25% of our population and therefore up to 25% of those in our study did not receive HBOT (2,3). A claim was made that therapeutic HBOT requires 30–40 treatments (3). This is problematic in general for both efficacy and effectiveness studies. Intention-to-treat treatment assignment, a hallmark of efficacy studies, is determined at the time of randomization and not after the patient has achieved a minimum exposure. Requiring 40 exposures could be part of a per protocol analysis but, based on the letters, would require a patient to receive several weeks of treatment. As a result, those who did not receive 40 exposures, even those that healed, would have to be removed from the analysis resulting in biased estimates. This seems unreasonable and unusual for a wound care study (7,8). There were concerns that too many subjects were excluded during the 4-week initiation phase of our study (about 57% of potentially eligible subjects were analyzed) (2). Exclusions were made prior to analysis and based on previous studies, mainly to exclude patients that presented with limbs that required immediate amputation, to exclude patients who presented for single visit consultations, and to exclude patients who healed in less than 4 weeks and would not have been eligible for HBOT. Notably, our exclusion percentage was similar to the 55% of eligible subjects analyzed in the RCT by Löndahl et al. (9). As noted in our title, HBOT was not used as a single therapy (2).
Finally, we never state that HBOT does not have physiologically helpful benefits and even cite the treatment of osteoradionecrosis of the jaw in which proper use including proper timing of HBOT is essential (4). As an analogy, many studies of cytokines that had strong physiological foundations were found not to be efficacious in RCTs (10). We never state that HBOT should not be used.
We strongly encourage others to plan new studies so that we can continue to shed more light on the effectiveness of HBOT. If our study along with efficacy studies like that by Löndahl et al. is correct, then it is vitally important for users of HBOT to carefully consider how they use HBOT and how they manage those that receive it (4,9). We look forward to reading new studies of HBOT.
Acknowledgments
No potential conflicts of interest relevant to this article were reported.
