External validation of prognostic models predicting pre-eclampsia: individual participant data meta-analysis
- PMID: 33131506
- PMCID: PMC7604970
- DOI: 10.1186/s12916-020-01766-9
External validation of prognostic models predicting pre-eclampsia: individual participant data meta-analysis
Abstract
Background: Pre-eclampsia is a leading cause of maternal and perinatal mortality and morbidity. Early identification of women at risk during pregnancy is required to plan management. Although there are many published prediction models for pre-eclampsia, few have been validated in external data. Our objective was to externally validate published prediction models for pre-eclampsia using individual participant data (IPD) from UK studies, to evaluate whether any of the models can accurately predict the condition when used within the UK healthcare setting.
Methods: IPD from 11 UK cohort studies (217,415 pregnant women) within the International Prediction of Pregnancy Complications (IPPIC) pre-eclampsia network contributed to external validation of published prediction models, identified by systematic review. Cohorts that measured all predictor variables in at least one of the identified models and reported pre-eclampsia as an outcome were included for validation. We reported the model predictive performance as discrimination (C-statistic), calibration (calibration plots, calibration slope, calibration-in-the-large), and net benefit. Performance measures were estimated separately in each available study and then, where possible, combined across studies in a random-effects meta-analysis.
Results: Of 131 published models, 67 provided the full model equation and 24 could be validated in 11 UK cohorts. Most of the models showed modest discrimination with summary C-statistics between 0.6 and 0.7. The calibration of the predicted compared to observed risk was generally poor for most models with observed calibration slopes less than 1, indicating that predictions were generally too extreme, although confidence intervals were wide. There was large between-study heterogeneity in each model's calibration-in-the-large, suggesting poor calibration of the predicted overall risk across populations. In a subset of models, the net benefit of using the models to inform clinical decisions appeared small and limited to probability thresholds between 5 and 7%.
Conclusions: The evaluated models had modest predictive performance, with key limitations such as poor calibration (likely due to overfitting in the original development datasets), substantial heterogeneity, and small net benefit across settings. The evidence to support the use of these prediction models for pre-eclampsia in clinical decision-making is limited. Any models that we could not validate should be examined in terms of their predictive performance, net benefit, and heterogeneity across multiple UK settings before consideration for use in practice.
Trial registration: PROSPERO ID: CRD42015029349 .
Keywords: External validation; Individual participant data; Pre-eclampsia; Prediction model.
Conflict of interest statement
The authors disclose support from NIHR HTA for the submitted work. LCC reports being Chair of the HTA CET Committee from January 2019. AK reports being a member of the NIHR HTA board. BWM reports grants from Merck; personal fees from OvsEva, Merck, and Guerbet; and other from NHMRC, Guerbet, and Merch, outside the submitted work. GS reports grants and personal fees from GlaxoSmithKline Research and Development Limited, grants from Sera Prognostics Inc., non-financial support from Illumina Inc., and personal fees and non-financial support from Roche Diagnostics Ltd., outside the submitted work. JK reports personal fees from Roche Canada, outside the submitted work. JM reports grants from National Health Research and Development Program, Health and Welfare Canada, during the conduct of the study. JEN reports grants from Chief Scientist Office Scotland, other from GlaxoSmithKline and Dilafor, outside the submitted work. AG reports personal fees from Roche Diagnostics, outside the submitted work. IH reports personal fees from Roche Diagnostics and Thermo Fisher, outside the submitted work. RR reports personal fees from the BMJ, Roche, and Universities of Leeds, Edinburgh, and Exeter, outside the submitted work.
Figures
Similar articles
-
Validation and development of models using clinical, biochemical and ultrasound markers for predicting pre-eclampsia: an individual participant data meta-analysis.Health Technol Assess. 2020 Dec;24(72):1-252. doi: 10.3310/hta24720. Health Technol Assess. 2020. PMID: 33336645 Free PMC article.
-
External validation of prognostic models to predict stillbirth using International Prediction of Pregnancy Complications (IPPIC) Network database: individual participant data meta-analysis.Ultrasound Obstet Gynecol. 2022 Feb;59(2):209-219. doi: 10.1002/uog.23757. Ultrasound Obstet Gynecol. 2022. PMID: 34405928
-
External validation, update and development of prediction models for pre-eclampsia using an Individual Participant Data (IPD) meta-analysis: the International Prediction of Pregnancy Complication Network (IPPIC pre-eclampsia) protocol.Diagn Progn Res. 2017 Oct 3;1:16. doi: 10.1186/s41512-017-0016-z. eCollection 2017. Diagn Progn Res. 2017. PMID: 31093545 Free PMC article.
-
Using Different Data Sets to Test How Well Clinical Prediction Models Work to Predict Patients' Risk of Heart Disease [Internet].Washington (DC): Patient-Centered Outcomes Research Institute (PCORI); 2021 Sep. Washington (DC): Patient-Centered Outcomes Research Institute (PCORI); 2021 Sep. PMID: 36848479 Free Books & Documents. Review.
-
Nontraditional Risk Factors in Cardiovascular Disease Risk Assessment: A Systematic Evidence Report for the U.S. Preventive Services Task Force [Internet].Rockville (MD): Agency for Healthcare Research and Quality (US); 2018 Jul. Report No.: 17-05225-EF-1. Rockville (MD): Agency for Healthcare Research and Quality (US); 2018 Jul. Report No.: 17-05225-EF-1. PMID: 30234933 Free Books & Documents. Review.
Cited by
-
Understanding the Pathophysiology of Preeclampsia: Exploring the Role of Antiphospholipid Antibodies and Future Directions.J Clin Med. 2024 May 2;13(9):2668. doi: 10.3390/jcm13092668. J Clin Med. 2024. PMID: 38731197 Free PMC article. Review.
-
Prognostic models for predicting clinical disease progression, worsening and activity in people with multiple sclerosis.Cochrane Database Syst Rev. 2023 Sep 8;9(9):CD013606. doi: 10.1002/14651858.CD013606.pub2. Cochrane Database Syst Rev. 2023. PMID: 37681561 Review.
-
Early Pregnancy Systolic Blood Pressure Patterns Predict Early- and Later-Onset Preeclampsia and Gestational Hypertension Among Ostensibly Low-to-Moderate Risk Groups.J Am Heart Assoc. 2023 Aug;12(15):e029617. doi: 10.1161/JAHA.123.029617. Epub 2023 Jul 12. J Am Heart Assoc. 2023. PMID: 37435795 Free PMC article.
-
Detecting Preeclampsia Severity Using Maternal-Obstetrical Characteristics and Complete Blood Cell Counts.Int J Gen Med. 2022 Dec 23;15:8715-8726. doi: 10.2147/IJGM.S392915. eCollection 2022. Int J Gen Med. 2022. PMID: 36582253 Free PMC article.
-
Prediction and prevention of preeclampsia by physicians in Brazil: An original study.Front Glob Womens Health. 2022 Oct 19;3:983131. doi: 10.3389/fgwh.2022.983131. eCollection 2022. Front Glob Womens Health. 2022. PMID: 36337683 Free PMC article.
References
-
- Cantwell R, Clutton-Brock T, Cooper G, Dawson A, Drife J, Garrod D, et al. Saving mothers’ lives: reviewing maternal deaths to make motherhood safer: 2006-2008. The Eighth Report of the Confidential Enquiries into Maternal Deaths in the United Kingdom. BJOG. 2011;118(Suppl 1):1–203. - PubMed
-
- Ng VK, Lo TK, Tsang HH, Lau WL, Leung WC. Intensive care unit admission of obstetric cases: a single centre experience with contemporary update. Hong Kong Med J. 2014;20(1):24–31. - PubMed
-
- Kleinrouweler CE, Cheong-See Mrcog FM, Collins GS, Kwee A, Thangaratinam S, Khan KS, et al. Prognostic models in obstetrics: available, but far from applicable. Am J Obstet Gynecol. 2016;214(1):79–90.e36. - PubMed
Publication types
MeSH terms
Grants and funding
LinkOut - more resources
Full Text Sources
Medical