Methodological Challenges for Risk Prediction in Perinatal Epidemiology

Purpose of ReviewTo discuss the methodological challenges in developing risk prediction models in perinatal epidemiology and barriers to their implementation in clinical practice.Recent FindingsIn perinatal epidemiology, risk prediction models have been created to examine the risk of adverse health outcomes in pregnancy, delivery, and post-partum periods. However, only a limited number of prediction models are being used to guide clinical decisions.SummaryThe accuracy and utility of prediction models for clinical decision making are contingent on the use of robust methods to develop risk prediction models and appropriate metrics to assess their performance and clinical impact. In order to increase the transportability (i.e., generalizability) of prediction models, careful consideration of the patient populations represented in the data used to develop and externally validate prediction models and the mechanism for data collection are needed. The era of big data provides researchers the opportunity to leverage existing databases, such as birth and pregnancy registries, through linkage to electronic health records, disease registries, and census data in order to enrich the breadth of clinical and sociodemographic information available for prediction modeling. However, these data sources introduce new challenges that require thorough assessment to evaluate their impact on the accuracy of resulting prediction models and their transportability to the general population.

[1]  Frank E. Harrell,et al.  Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis , 2001 .

[2]  Yvonne Vergouwe,et al.  Prognosis and prognostic research: what, why, and how? , 2009, BMJ : British Medical Journal.

[3]  R. Tibshirani,et al.  Improvements on Cross-Validation: The 632+ Bootstrap Method , 1997 .

[4]  John P. A. Ioannidis,et al.  Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review , 2017, J. Am. Medical Informatics Assoc..

[5]  E. Steyerberg,et al.  Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research , 2013, PLoS medicine.

[6]  Daniel B. Mark,et al.  TUTORIAL IN BIOSTATISTICS MULTIVARIABLE PROGNOSTIC MODELS: ISSUES IN DEVELOPING MODELS, EVALUATING ASSUMPTIONS AND ADEQUACY, AND MEASURING AND REDUCING ERRORS , 1996 .

[7]  J. Kassirer,et al.  The threshold approach to clinical decision making. , 1980, The New England journal of medicine.

[8]  J. Hutcheon,et al.  Assessment of the fullPIERS Risk Prediction Model in Women With Early-Onset Preeclampsia , 2018, Hypertension.

[9]  M. Ezzati,et al.  Major risk factors for stillbirth in high-income countries: a systematic review and meta-analysis , 2011, The Lancet.

[10]  N. Cook Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction , 2007, Circulation.

[11]  Ewout W. Steyerberg,et al.  Big Data and Predictive Analytics: Recalibrating Expectations , 2018, JAMA.

[12]  Richard D Riley,et al.  External validation of clinical prediction models using big datasets from e-health records or IPD meta-analysis: opportunities and challenges , 2016, BMJ.

[13]  Hester F. Lingsma,et al.  Assessing the incremental value of diagnostic and prognostic markers: a review and illustration , 2012, European journal of clinical investigation.

[14]  Holly Janes,et al.  Assessing the Value of Risk Predictions by Using Risk Stratification Tables , 2008, Annals of Internal Medicine.

[15]  E. Hall,et al.  Risk Prediction for Adverse Pregnancy Outcomes in a Medicaid Population. , 2015, Journal of women's health.

[16]  Patrick Royston,et al.  Multiple imputation using chained equations: Issues and guidance for practice , 2011, Statistics in medicine.

[17]  Chao Yan,et al.  Predicting Neonatal Encephalopathy From Maternal Data in Electronic Medical Records , 2018, AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science.

[18]  G. Bedogni,et al.  Clinical Prediction Models—a Practical Approach to Development, Validation and Updating , 2009 .

[19]  Ewout W Steyerberg,et al.  Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests , 2016, British Medical Journal.

[20]  C B Begg,et al.  A General Regression Methodology for ROC Curve Estimation , 1988, Medical decision making : an international journal of the Society for Medical Decision Making.

[21]  T. Garite,et al.  Fetal fibronectin: a new tool for the prediction of successful induction of labor. , 1996, American journal of obstetrics and gynecology.

[22]  A. Maas,et al.  Cardiovascular risk management after reproductive and pregnancy-related disorders: A Dutch multidisciplinary evidence-based guideline. , 2015, European journal of preventive cardiology.

[23]  Z. Bhutta,et al.  Development and internal validation of a multivariable model to predict perinatal death in pregnancy hypertension. , 2015, Pregnancy hypertension.

[24]  S. Galea,et al.  Population Health Science and the Challenges of Prediction , 2017, Annals of Internal Medicine.

[25]  G. Colditz,et al.  A stillbirth calculator: Development and internal validation of a clinical prediction model to quantify stillbirth risk , 2017, PloS one.

[26]  Dwight J Rouse,et al.  Development of a Nomogram for Prediction of Vaginal Birth After Cesarean Delivery , 2007, Obstetrics and gynecology.

[27]  E. Elkin,et al.  Decision Curve Analysis: A Novel Method for Evaluating Prediction Models , 2006, Medical decision making : an international journal of the Society for Medical Decision Making.

[28]  J. Hutcheon,et al.  Prediction of adverse maternal outcomes from pre-eclampsia and other hypertensive disorders of pregnancy: A systematic review. , 2017, Pregnancy hypertension.

[29]  Patrick Royston,et al.  A new measure of prognostic separation in survival data , 2004, Statistics in medicine.

[30]  F van der Veen,et al.  Evaluating prediction models in reproductive medicine. , 2009, Human reproduction.

[31]  F. Harrell,et al.  Prognostic/Clinical Prediction Models: Multivariable Prognostic Models: Issues in Developing Models, Evaluating Assumptions and Adequacy, and Measuring and Reducing Errors , 2005 .

[32]  G. Visser,et al.  Indicators for intervention during the expulsive second‐stage arrest of labour , 2009, BJOG : an international journal of obstetrics and gynaecology.

[33]  Donna M. Zulman,et al.  The relative merits of population-based and targeted prevention strategies. , 2008, The Milbank quarterly.

[34]  M. Kuzniewicz Management der Early-onset-Sepsis , 2017, Zeitschrift für Geburtshilfe und Neonatologie.

[35]  James E. Helmreich Regression Modeling Strategies with Applications to Linear Models, Logistic and Ordinal Regression and Survival Analysis (2nd Edition) , 2016 .

[36]  K. Nicolaides,et al.  Prediction of small‐for‐gestational‐age neonates: screening by biophysical and biochemical markers at 30–34 weeks , 2015, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[37]  C. Parker,et al.  Association Between Stillbirth and Illicit Drug Use and Smoking During Pregnancy , 2014, Obstetrics and gynecology.

[38]  N. Obuchowski,et al.  Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures , 2010, Epidemiology.

[39]  Maarten van Smeden,et al.  Sample size for binary logistic prediction models: Beyond events per variable criteria , 2018, Statistical methods in medical research.

[40]  Patrick Royston,et al.  Simplifying a prognostic model: a simulation study based on clinical data , 2002, Statistics in medicine.

[41]  M. Pepe,et al.  Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. , 2004, American journal of epidemiology.

[42]  P. Kearney,et al.  Association of Hypertensive Disorders of Pregnancy With Risk of Neurodevelopmental Disorders in Offspring: A Systematic Review and Meta-analysis , 2018, JAMA psychiatry.

[43]  W. Kannel,et al.  A general cardiovascular risk profile: the Framingham Study. , 1976, The American journal of cardiology.

[44]  A. Keir,et al.  A quantitative, risk‐based approach to the management of neonatal early‐onset sepsis , 2018, Acta paediatrica.

[45]  Hang Qiu,et al.  Electronic Health Record Driven Prediction for Gestational Diabetes Mellitus in Early Pregnancy , 2017, Scientific Reports.

[46]  M. Pencina,et al.  Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation , 2004, Statistics in medicine.

[47]  P. Royston,et al.  External validation of a Cox prognostic model: principles and methods , 2013, BMC Medical Research Methodology.

[48]  Shoo K. Lee,et al.  Prediction of Neonatal Outcomes in Extremely Preterm Neonates , 2013, Pediatrics.

[49]  M. Schumacher,et al.  Consistent Estimation of the Expected Brier Score in General Survival Models with Right‐Censored Event Times , 2006, Biometrical journal. Biometrische Zeitschrift.

[50]  M. Woodward,et al.  Risk prediction models: II. External validation, model updating, and impact assessment , 2012, Heart.

[51]  J. Robins,et al.  Correcting for Noncompliance and Dependent Censoring in an AIDS Clinical Trial with Inverse Probability of Censoring Weighted (IPCW) Log‐Rank Tests , 2000, Biometrics.

[52]  M. Pencina,et al.  Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond , 2008, Statistics in medicine.

[53]  J. Hutcheon,et al.  Prediction of adverse maternal outcomes in pre-eclampsia: development and validation of the fullPIERS model , 2011, The Lancet.

[54]  Gowri Raman,et al.  Tufts PACE Clinical Predictive Model Registry: update 1990 through 2015 , 2017, Diagnostic and Prognostic Research.

[55]  Accuracy of Cardiovascular Risk Prediction Varies by Neighborhood Socioeconomic Position , 2018, Annals of Internal Medicine.