Developing and Validating a Risk Score for Lower-Extremity Amputation in Patients Hospitalized for a Diabetic Foot Infection

We used data from a clinical research database of patients hospitalized at 97 acute-care hospitals in the U.S. that was compiled by CareFusion (Department of Clinical Research, CareFusion, Marlborough, MA). The database includes extensive data in the following categories: clinical (including diagnoses and vital signs); laboratory (e.g., chemistry, hematology, and microbiology); and administrative (e.g., demographics, admission source, length of hospitalization, and discharge status). Eligible patients were those discharged from one of the designated hospitals between 1 January 2003 and 30 June 2007 with a principal diagnosis (ICD-9-CM) of diabetes and a secondary diagnosis indicating skin or soft-tissue infection (including cellulitis, infected ulcer, or surgical-site infection [SSI]) that was culture documented within 48 h of admission. This study was approved by the New England Institutional Review Board Human Subjects Research Committee (Wellesley, MA) and was conducted in compliance with the Health Insurance Portability and Accountability Act.

We randomly split the study population into two groups: one for model derivation (75% of the population) and the other for model validation (the remaining 25%). Candidate predictor variables used in the model were demographic data, a history of previous health care–associated infection, clinical presentation, concomitant chronic disease(s), previous LEA, and type of skin or soft-tissue infection (cellulitis, ulcer or other infection, or SSI). Our outcome measure of interest was LEA during the index hospitalization, which was identified by ICD-9 procedure codes (841.1 amputation, toe; 841.2 amputation, foot; 841.3 disarticulation, ankle; 841.4 amputation, Malleoli; 841.5 amputation, knee, below NEC [not elsewhere classified]; 841.6 disarticulation, knee; and 841.7 amputation, knee, above).

Using the derivation cohort, we conducted univariate analyses to determine the proportion of patients with each candidate predictor who underwent an LEA. We then fit a stepwise multivariate logistic regression model to identify independent predictors of LEA and to estimate their relative predictive weights (coefficients). Using previously published methods, we converted the coefficients for the independent predictors into a simplified risk score system (11). Specifically, we calculated the number of points assigned to each variable by dividing its regression coefficient by the smallest coefficient in the model then rounded this quotient to the nearest whole number. We then calculated each subject’s LEA risk score by summing up the points of all variables present on admission. We then validated the risk score system using the remaining 25% of the population.

We assessed model discrimination using the c-statistic, which defines how well a model or prediction rule can discriminate between patients who do and do not have an event and measures how well a clinical prediction rule correctly ranks patients in order by risk. We assessed model calibration using the Hosmer-Lemeshow goodness-of-fit test, which assesses whether the observed and expected event rates match in subgroups of the model population. The test specifically identifies subgroups as the deciles of fitted risk values, and models with similar expected and observed event rates (i.e., a large P value) are considered to be well calibrated. We then used the Cochran-Armitage trending statistic, which modifies the χ² test to incorporate a suspected ordering, to assess the ability of the risk score system to differentiate low-risk from high-risk patients in a graded response. All analyses were conducted using Statistical Analysis System (version 9.01; SAS Institute, Cary, NC).

Among hospitalized patients, 3,018 met our inclusion criteria for a diabetic foot infection, with cellulitis (in 80%) and an infected ulcer (16%) being the most common diagnoses. We used 2,230 patients for the derivation cohort and 788 for the validation cohort; all baseline characteristics were similar between the two cohorts (Table 1). A total of 646 (21.4%) patients underwent an LEA during their index hospitalization; the number (and rate) in the derivation cohort was 463 (20.8%) compared with 183 (23.2%) for the validation cohort. For those patients undergoing an LEA, the median time from admission to amputation was 4 days, with an interquartile range of 2–7 days. A previous LEA of some type was noted in ~27% of patients in the derivation cohort and 26% in the validation cohort.

For the entire study cohort, the patients who underwent an LEA were significantly older (median age in years [interquartile range] 62 [53–72] vs. 60 [50–71]; P < 0.0001), and their in-hospital mortality rate was significantly higher (2.3 vs. 1.1%; P < 0.05) compared with patients who did not require amputation. The most common finding on culture was a polymicrobial (two or more different microorganisms) infection, which accounted for ~57% of all patients. A detailed accounting of pathogen distribution is shown in Supplementary Appendix A.

As shown in Table 2, in the derivation cohort the univariate analysis revealed that the following factors were significantly associated with LEA (P < 0.05): older age; male sex; transfer from another hospital or nursing home; previous LEA; coronary, renal, or peripheral vascular disease; low serum albumin; elevated values for white blood cell count, prothrombin time or international normalized ratio, or creatinine; elevated body temperature; the presence of a foot ulcer; and the presence of an SSI.

Using stepwise regression analysis, we found 11 independent predictors of LEA (Table 3). The most highly significant (P < 0.0001) were SSI, vasculopathy, previous LEA, and white blood cell count >11,000 per mm³. The predictive model developed using these predictors had very good discrimination (c-statistic 0.76) and excellent calibration between predicted and observed LEA rates (Hosmer-Lemeshow test showing that they did not significantly differ across risk deciles; P = 0.63) (Fig. 1). Patients in the highest risk decile had a predicted probability of LEA of 59.4% and an observed LEA rate of 58.7%, whereas those in the lowest decile had a predicted probability of LEA of 4% and an observed LEA rate of 5%. The predictive model yielded a good calibration when applied to the validation cohort (Hosmer-Lemeshow test; P = 0.33).

For each patient, we summed all the variables present on admission to create an LEA risk score. We then grouped the risk scores into five risk strata. The use of a five-level risk strata allows easy application of risk stratification for comparisons of outcomes. LEA rates in the derivation and validation cohorts increased significantly by risk score strata (P < 0.0001 by the Cochran-Armitage trending test for both derivation and validation cohorts) (Fig. 2). For the derivation cohort, the risk of LEA for patients aged <50 years and without any of the other 10 factors in the risk score system was essentially zero. In contrast, for those whose score was ≥21, LEA risk was ~50%. The findings in the validation cohort were similar to those in the derivation cohort.

In this large cohort of patients hospitalized for a diabetic foot infection, more than one-fifth required an LEA. In reviewing numerous clinical and laboratory variables present at hospital admission in our derivation cohort, we identified 11 significant independent risk factors for LEA. By rounding the logistic regression coefficients into integers, we developed a simple LEA risk score system with five strata that we demonstrated was highly predictive of the risk for LEA. Using the patients in the validation cohort, we were then able to demonstrate that this risk score was indeed valid and well calibrated.

Most of the factors included in our risk score have been reported as risks for LEA in smaller, previously published studies (7,12,13). The presence of infection and peripheral vascular disease are the most powerful predictors. Most patients with diabetic foot ulcers do not have a fever or leukocytosis (14), which define a severe infection according to the Infectious Diseases Society of America and the International Working Group on the Diabetic Foot criteria. These severe diabetic foot infections are associated with a greater risk of LEA than those of mild or moderate severity. Previous studies (15,17) also have identified renal insufficiency as being associated with an increased risk of amputations. Other studies have identified increasing age (18), male sex (16), and hypoalbuminemia (19) as risks for LEA. Likewise, a previous LEA is a strong risk factor predicting the need for another amputation (12). In none of these previous studies, however, did the authors attempt to construct a scoring system to predict amputation risk for both men and women.

In a previous study (20), we showed that patients with SSIs who were transferred from another acute-care facility had worse clinical and economic outcomes, perhaps because patients with infections of greater severity are more likely to be transferred to hospitals with more intensive resources or greater expertise. A recent meta-analysis (21) demonstrated a direct association between hyperglycemia (as measured by hemoglobin A_1c) and LEA. Unfortunately, we did not have hemoglobin A_1c values on most of our patients, but we did note a nonsignificant trend toward higher amputation rates in those with increased blood glucose levels on admission. All of our patients had an infection; therefore, that variable was not among those included in our scoring system. It is noteworthy that the type of infection was associated with amputation risk, with SSIs at the highest risk, followed by infected ulcers, when compared with cellulitis. Our assumption is that these SSIs may be associated with failed lower-extremity bypass procedures. We found no other studies that investigated the risk of adverse outcomes in patients with an infected ulcer compared with cellulitis or SSIs. We did find other studies (7,14,22,23) that reported that deep infections (especially those involving bone) and necrotizing infections more often resulted in amputations.

The simplified five risk strata that we devised correlated strongly with LEA rates. This may have important clinical implications on how to allocate resources. In particular, a patient with a low score may need fewer medical resources than a patient with a high LEA risk score. At the other extreme, to try to avoid the tragedy of amputation, health care providers should concentrate efforts on a patient with a risk score of >21, who has a 50% chance of an LEA. Our finding that patients transferred from another acute-care hospital had the highest odds ratio for LEA highlights the need of risk adjustment to appropriately evaluate outcomes for hospitals treating the most severe patients. Because LEA rates are sometimes used to compare quality of care for patients with diabetic foot complications, our risk adjustment score could be used to ensure that centers treating higher-risk patients are not unfairly penalized. Furthermore, although this has not been tested, the score might be helpful to clinicians in deciding which patients with diabetic foot infections may need to be hospitalized. The LEA risk score system has the benefit of being simple to use; each of the risk factors is readily available, usually at the time of admission or soon afterward.

Our study is limited by the fact that our analysis was retrospective, and, although fairly inclusive, we could have missed potentially significant factors. For example, the individual reasons for amputation and whether amputation was elective or urgent were not captured in the database. One major risk factor that we did not capture that could have an effect on our risk score is a history of a previous lower-extremity revascularization procedure, which was a significant factor in other reports (24,25). Selection bias is another potential limitation when using administrative data to identify patients with skin or soft-tissue infections. To minimize potential bias related to the use of ICD-9 coding, we limited our study to culture-confirmed infections.

In conclusion, we used a large clinical database to develop and validate a risk score that seems to accurately stratify LEA risk among patients hospitalized for a diabetic foot infection. This score may help clinicians identify patients at highest risk of LEA upon admission. Once patient identification is achieved, methods to reduce the risk can be investigated. We would like to see our risk score validated prospectively, including in patients treated on an outpatient basis.

This project was funded in part by the Ortho-McNeil Janssen Scientific Affairs (Raritan, NJ). B.A.L. serves as a consultant to the Ortho-McNeil Janssen Scientific Affairs. X.S., R.S.J., K.G.D., and Y.P.T. are employees of CareFusion. No other potential conflicts of interest relevant to this article were reported.

B.A.L. researched data, contributed to discussion, and wrote, reviewed, and edited the manuscript. J.A.W. contributed to discussion and wrote, reviewed, and edited the manuscript. X.S., R.S.J., and K.G.D. researched data, contributed to discussion, and reviewed and edited the manuscript. Y.P.T. researched data, contributed to discussion, and wrote, reviewed, and edited the manuscript.

Parts of this study were presented in abstract form at the 47th Annual Meeting of the Infectious Diseases Society of America, Philadelphia, Pennsylvania, 29 October to 1 November 2009.