Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Oct 14;8(4):e001388.
doi: 10.1136/bmjsem-2022-001388. eCollection 2022.

Predictive models for musculoskeletal injury risk: why statistical approach makes all the difference

Daniel I Rhon  1   2 , Deydre S Teyhen  3 , Gary S Collins  4   5 , Garrett S Bullock  6   7
Affiliations

Predictive models for musculoskeletal injury risk: why statistical approach makes all the difference

Daniel I Rhon et al. BMJ Open Sport Exerc Med. .

Abstract

Objective: Compare performance between an injury prediction model categorising predictors and one that did not and compare a selection of predictors based on univariate significance versus assessing non-linear relationships.

Methods: Validation and replication of a previously developed injury prediction model in a cohort of 1466 service members followed for 1 year after physical performance, medical history and sociodemographic variables were collected. The original model dichotomised 11 predictors. The second model (M2) kept predictors continuous but assumed linearity and the third model (M3) conducted non-linear transformations. The fourth model (M4) chose predictors the proper way (clinical reasoning and supporting evidence). Model performance was assessed with R2, calibration in the large, calibration slope and discrimination. Decision curve analyses were performed with risk thresholds from 0.25 to 0.50.

Results: 478 personnel sustained an injury. The original model demonstrated poorer R2 (original:0.07; M2:0.63; M3:0.64; M4:0.08), calibration in the large (original:-0.11 (95% CI -0.22 to 0.00); M2: -0.02 (95% CI -0.17 to 0.13); M3:0.03 (95% CI -0.13 to 0.19); M4: -0.13 (95% CI -0.25 to -0.01)), calibration slope (original:0.84 (95% CI 0.61 to 1.07); M2:0.97 (95% CI 0.86 to 1.08); M3:0.90 (95% CI 0.75 to 1.05); M4: 081 (95% CI 0.59 to 1.03) and discrimination (original:0.63 (95% CI 0.60 to 0.66); M2:0.90 (95% CI 0.88 to 0.92); M3:0.90 (95% CI 0.88 to 0.92); M4: 0.63 (95% CI 0.60 to 0.66)). At 0.25 injury risk, M2 and M3 demonstrated a 0.43 net benefit improvement. At 0.50 injury risk, M2 and M3 demonstrated a 0.33 net benefit improvement compared with the original model.

Conclusion: Model performance was substantially worse in the models with dichotomised variables. This highlights the need to follow established recommendations when developing prediction models.

Keywords: Injury; Neuromuscular; Prevention; Risk factor.

PubMed Disclaimer

Conflict of interest statement

Competing interests: None declared.

Figures

Figure 1
Figure 1
Calibration slope of original developed model (M1). Calibration is the relationship between predicted and actual probability of the event. The calibration slope plots the predicted risk graphically against the observed outcome; displaying the calibration intercept and calibration slope. perfect calibration would result in a 45° line. Within this calibration plot, risk does not begin until 0.20. What this clinically means is that everyone’s risk cannot be lower than 0.20. Individuals with little risk of injury may be inappropriately referred for clinical care.
Figure 2
Figure 2
Calibration slope of original developed model replicated with continuous predictors; linearity assumed (M2). Calibration is the relationship between predicted and actual probability of the event. The calibration slope plots the predicted risk graphically against the observed outcome; displaying the calibration intercept and calibration slope. perfect calibration would result in a 45° line. The predicted risk is lower between 0.20 and 0.40, compared with the actual risk for these individuals. What this means clinically is that these individuals would be under estimated for their true risk of injury, and potentially not referred to the appropriate clinical care or injury prevention strategies.
Figure 3
Figure 3
Calibration slope of original developed model replicated with continuous predictors and non-linear transformations (M3). Calibration is the relationship between predicted and actual probability of the event. The calibration slope plots the predicted risk graphically against the observed outcome; displaying the calibration intercept and calibration slope. Perfect calibration would result in a 45° line. Predicted risk between 0.20 and 0.40 is slightly lower than actual risk, which may alter clinical decisions for individuals within this risk threshold.
Figure 4
Figure 4
Calibration slope when using best practices to choose predictors (added several new predictors), but keeping them all dichotomised (M4). Calibration is the relationship between predicted and actual probability of the event. The calibration slope plots the predicted risk graphically against the observed outcome; displaying the calibration intercept and calibration slope. Perfect calibration would result in a 45° line. Within this calibration plot, risk does not begin until 0.20. What this clinically means is that everyone’s risk cannot be lower than 0.20. Individuals with little risk of injury may be inappropriately referred for clinical care.
Figure 5
Figure 5
Calibration slope of optimal model developed based on appropriately chosen predictors, keeping predictors continuous and conducting appropriate non-linear transformations (M5). Calibration is the relationship between predicted and actual probability of the event. Perfect calibration would result in a 45° line. The calibration slope plots the predicted risk graphically against the observed outcome; displaying the calibration intercept and calibration slope. This calibration model demonstrates risk from 0.00 to 1.0, and has the most uniform predicted risk to the actual risk. Predicted risk between 0.20 and 0.40 is lower than actual risk, which may alter clinical decisions for individuals with the least risk of injury.
Figure 6
Figure 6
Decision curve for the prediction models to predict injury risk in military personnel. The figure reports the expected net benefit compared with not predicting injuries. ‘Treat all’ assumes that all military personnel are at a high risk for injury and should be intervened on, while treat none assumes that all military personnel are at a low risk for injury and NO interventions should be performed. The threshold probability was defined as the population risk of injury within military personnel of 0.25–0.50. The models keeping predictors continuous (M2, M3, M5) non-linear transformations (M3, M5) all demonstrated improved net benefit (ie, correct injury identification) compared with ‘treat all’ and the original (M1) model and the original model with further added dichotomised predictors (M4) at these threshold probabilities.
Figure 7
Figure 7
Example of the multivariate injury dynamic nomogram. Probability of injury is approximately 22.5% (95% CI of 10.6% to 41.8%) when a 24-year-old male has one previous injury in the military, a recovery score of 70, was on profile the previous year, five movements reported pain, a 5° asymmetry in ankle dorsiflexion, 68% limb length on the Y-Balance anterior reach, 68% limb length on the Y-Balance upper quarter superolateral reach, and 7% limb length asymmetry on the Y-Balance upper quarter inferolateral reach and completes the 2 mile run in a time in 868 s (14 min, 29 s). (More examples in online supplemental appendix).
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Sebbag E, Felten R, Sagez F, et al. . The world-wide burden of musculoskeletal diseases: a systematic analysis of the world Health organization burden of diseases database. Ann Rheum Dis 2019;78:844–8. 10.1136/annrheumdis-2019-215142 - DOI - PubMed
    1. Bittencourt NFN, Meeuwisse WH, Mendonça LD, et al. . Complex systems approach for sports injuries: moving from risk factor identification to injury pattern recognition—narrative review and new concept. Br J Sports Med 2016;50:1309–14. 10.1136/bjsports-2015-095850 - DOI - PubMed
    1. Stern BD, Hegedus EJ, Lai Y-C. Injury prediction as a non-linear system. Phys Ther Sport 2020;41:43–8. 10.1016/j.ptsp.2019.10.010 - DOI - PubMed
    1. Steyerberg EW, Uno H, Ioannidis JPA, et al. . Poor performance of clinical prediction models: the harm of commonly applied methods. J Clin Epidemiol 2018;98:133–43. 10.1016/j.jclinepi.2017.11.013 - DOI - PubMed
    1. Sainani KL, Borg DN, Caldwell AR, et al. . Call to increase statistical collaboration in sports science, sport and exercise medicine and sports physiotherapy. Br J Sports Med 2021;55:118–22. 10.1136/bjsports-2020-102607 - DOI - PMC - PubMed

LinkOut - more resources