Standards should be applied in the prevention and handling of missing data for patient-centered outcomes research: a systematic review and expert consensus.

OBJECTIVES To recommend methodological standards in the prevention and handling of missing data for primary patient-centered outcomes research (PCOR). STUDY DESIGN AND SETTING We searched National Library of Medicine Bookshelf and Catalog as well as regulatory agencies' and organizations' Web sites in January 2012 for guidance documents that had formal recommendations regarding missing data. We extracted the characteristics of included guidance documents and recommendations. Using a two-round modified Delphi survey, a multidisciplinary panel proposed mandatory standards on the prevention and handling of missing data for PCOR. RESULTS We identified 1,790 records and assessed 30 as having relevant recommendations. We proposed 10 standards as mandatory, covering three domains. First, the single best approach is to prospectively prevent missing data occurrence. Second, use of valid statistical methods that properly reflect multiple sources of uncertainty is critical when analyzing missing data. Third, transparent and thorough reporting of missing data allows readers to judge the validity of the findings. CONCLUSION We urge researchers to adopt rigorous methodology and promote good science by applying best practices to the prevention and handling of missing data. Developing guidance on the prevention and handling of missing data for observational studies and studies that use existing records is a priority for future research.

[1]  A. Barton Handbook for good clinical research practice (GCP): guidance for implementation , 2007, Journal of Epidemiology and Community Health.

[2]  R. Guy,et al.  International Conference on Harmonisation , 2014 .

[3]  Michael L. Johnson,et al.  Good research practices for comparative effectiveness research: analytic methods to improve causal inference from nonrandomized studies of treatment effects using secondary data sources: the ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report--Part III. , 2009, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[4]  C D Naylor,et al.  Dealing with missing data in observational health care outcome analyses. , 2000, Journal of clinical epidemiology.

[5]  Paula Diehr,et al.  Imputation of missing longitudinal data: a comparison of methods. , 2003, Journal of clinical epidemiology.

[6]  S. Greenfield,et al.  COMMITTEE ON STANDARDS FOR DEVELOPING TRUSTWORTHY CLINICAL PRACTICE GUIDELINES , 2011 .

[7]  Mirjam Kretzschmar,et al.  Dynamic Transmission Modeling , 2012, Medical decision making : an international journal of the Society for Medical Decision Making.

[8]  R. Kay Statistical Principles for Clinical Trials , 1998, The Journal of international medical research.

[9]  Meera Viswanathan,et al.  Comparative Effectiveness Review Methods: Clinical Heterogeneity , 2010 .

[10]  A. Carr,et al.  Primary total hip replacement surgery: a systematic review of outcomes and modelling of cost-effectiveness associated with different prostheses. , 1998, Health technology assessment.

[11]  Matthias Egger,et al.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies , 2007, PLoS medicine.

[12]  Uwe Siebert,et al.  Good research practices for comparative effectiveness research: approaches to mitigate bias and confounding in the design of nonrandomized studies of treatment effects using secondary data sources: the International Society for Pharmacoeconomics and Outcomes Research Good Research Practices for Retr , 2009, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[13]  N. Dreyer,et al.  Registries for Evaluating Patient Outcomes: A User’s Guide , 2010 .

[14]  S. Pocock,et al.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. , 2007, Preventive medicine.

[15]  Zhang Jing,et al.  Guideline on missing data in confirmatory clinical trials , 2012 .

[16]  McGinnis Jm,et al.  The learning healthcare system : workshop summary , 2007 .

[17]  A. Dobson,et al.  RE: “QUALITY OF REPORTING OF OBSERVATIONAL LONGITUDINAL RESEARCH” , 2005 .

[18]  Haitao Chu,et al.  On estimation of vaccine efficacy using validation samples with selection bias. , 2006, Biostatistics.

[19]  R. Willke,et al.  Good research practices for cost-effectiveness analysis alongside clinical trials: the ISPOR RCT-CEA Task Force report. , 2005, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[20]  Jonathan Karnon,et al.  Modeling using discrete event simulation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force--4. , 2012, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[21]  LeighAnne Olsen,et al.  IOM Roundtable on Evidence-Based Medicine , 2007 .

[22]  D. Moher,et al.  CONSORT 2010 Statement: updated guidelines for reporting parallel group randomized trials , 2010, Obstetrics and gynecology.

[23]  Global Sensitivity Analysis for Randomized Trials with Informative Dropout: A Semiparametric Perspective , 2009 .

[24]  Laura A. Levit,et al.  Finding what works in health care : standards for systematic reviews , 2011 .

[25]  R. Little,et al.  The design and conduct of clinical trials to limit missing data , 2012, Statistics in medicine.

[26]  D. Rubin Multiple imputation for nonresponse in surveys , 1989 .

[27]  L. Garrison,et al.  Using real-world data for coverage and payment decisions: the ISPOR Real-World Data Task Force report. , 2007, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[28]  T. Stijnen,et al.  Review: a gentle introduction to imputation of missing values. , 2006, Journal of clinical epidemiology.

[29]  Joseph C Cappelleri,et al.  Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1. , 2011, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[30]  A. Rotnitzky,et al.  Missing Data in Longitudinal Studies: Strategies for Bayesian Modeling and Sensitivity Analysis by DANIELS, M. J. and HOGAN, J. W , 2009 .

[31]  D. Moher,et al.  CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials , 2010, Journal of clinical epidemiology.

[32]  R. Little,et al.  The prevention and treatment of missing data in clinical trials. , 2012, The New England journal of medicine.

[33]  D Scharfstein,et al.  Methods for Conducting Sensitivity Analysis of Trials with Potentially Nonignorable Competing Causes of Censoring , 2001, Biometrics.

[34]  Sharon-Lise Normand,et al.  Prospective observational studies to assess comparative effectiveness: the ISPOR good research practices task force report. , 2012, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[35]  S Greenland,et al.  A critical look at methods for handling missing covariates in epidemiologic regression analyses. , 1995, American journal of epidemiology.

[36]  Qingxia Chen,et al.  Missing covariate data in medical research: to impute is better than to ignore. , 2010, Journal of clinical epidemiology.

[37]  T. Fleming Addressing Missing Data in Clinical Trials , 2011, Annals of Internal Medicine.

[38]  John P. A. Ioannidis,et al.  Methodological standards and patient-centeredness in comparative effectiveness research: the PCORI perspective. , 2012, JAMA.

[39]  Ian R White,et al.  Are missing outcome data adequately handled? A review of published randomized controlled trials in major medical journals , 2004, Clinical trials.

[40]  D. Rubin,et al.  Statistical Analysis with Missing Data. , 1989 .

[41]  Joseph W Hogan,et al.  Handling drop‐out in longitudinal studies , 2004, Statistics in medicine.

[42]  Sharon-Lise T Normand,et al.  Getting the methods right--the foundation of patient-centered outcomes research. , 2012, The New England journal of medicine.

[43]  D. Spiegelhalter,et al.  Consensus development methods, and their use in clinical guideline development. , 1998, Health technology assessment.

[44]  Peter Davey,et al.  A checklist for retrospective database studies--report of the ISPOR Task Force on Retrospective Databases. , 2003, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.