Limitations in Using Multiple Imputation to Harmonize Individual Participant Data for Meta-Analysis

Individual participant data (IPD) meta-analysis is a meta-analysis in which the individual-level data for each study are obtained and used for synthesis. A common challenge in IPD meta-analysis is when variables of interest are measured differently in different studies. The term harmonization has been coined to describe the procedure of placing variables on the same scale in order to permit pooling of data from a large number of studies. Using data from an IPD meta-analysis of 19 adolescent depression trials, we describe a multiple imputation approach for harmonizing 10 depression measures across the 19 trials by treating those depression measures that were not used in a study as missing data. We then apply diagnostics to address the fit of our imputation model. Even after reducing the scale of our application, we were still unable to produce accurate imputations of the missing values. We describe those features of the data that made it difficult to harmonize the depression measures and provide some guidelines for using multiple imputation for harmonization in IPD meta-analysis.

[1]  J. Matson,et al.  The assessment of depression in children: the internal structure of the Child Depression Inventory (CDI). , 1984, Behaviour research and therapy.

[2]  Ahnalee M. Brincks,et al.  Two-Year Impact of Prevention Programs on Adolescent Depression: an Integrative Data Analysis Approach , 2018, Prevention Science.

[3]  Robert D Gibbons,et al.  Multiple imputation for harmonizing longitudinal non‐commensurate measures in individual participant data meta‐analysis , 2015, Statistics in medicine.

[4]  Robert D. Gibbons,et al.  Children's Depression Rating Scale--Revised , 2017 .

[5]  Lei Liu,et al.  Testing moderation in network meta‐analysis with individual participant data , 2016, Statistics in medicine.

[6]  N. Sartorius,et al.  Assessment of Depression , 1986, Springer Berlin Heidelberg.

[7]  M. Hamilton A RATING SCALE FOR DEPRESSION , 1960, Journal of neurology, neurosurgery, and psychiatry.

[8]  Wenjing Huang,et al.  Pooling data from multiple longitudinal studies: the role of item response theory in integrative data analysis. , 2008, Developmental psychology.

[9]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[10]  P. Lewinsohn,et al.  A self- and parent-report measure of adolescent depression: The Child Behavior Checklist Depression scale (CBCL-D). , 1992 .

[11]  J. Schafer,et al.  Computational Strategies for Multivariate Linear Mixed-Effects Models With Missing Values , 2002 .

[12]  N. Nevin-Folino Advancing Science Through Collaborative NICU Nutrition , 2014 .

[13]  T. Achenbach Manual for the child behavior checklist/4-18 and 1991 profile , 1991 .

[14]  Matthieu Resche-Rigon,et al.  Multiple imputation for handling systematically missing confounders in meta‐analysis of individual participant data , 2013, Statistics in medicine.

[15]  J. Weisz,et al.  The Youth Self Report: Applicability and Validity Across Younger and Older Youths , 2011, Journal of clinical child and adolescent psychology : the official journal for the Society of Clinical Child and Adolescent Psychology, American Psychological Association, Division 53.

[16]  John Geweke,et al.  Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments , 1991 .

[17]  I. Bernstein,et al.  Psychometric properties of the Children's Depression Rating Scale-Revised in adolescents. , 2010, Journal of child and adolescent psychopharmacology.

[18]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[19]  Andrea M Hussong,et al.  Integrative data analysis: the simultaneous analysis of multiple data sets. , 2009, Psychological methods.

[20]  B. Carlin,et al.  Markov Chain Monte Carlo conver-gence diagnostics: a comparative review , 1996 .

[21]  Patrick J Curran,et al.  The seemingly quixotic pursuit of a cumulative psychological science: introduction to the special issue. , 2009, Psychological methods.

[22]  Roger Tarling,et al.  Multiple Imputation for handling missing data in social research , 2014 .

[23]  Daniel J Bauer,et al.  Integrative data analysis in clinical psychology research. , 2013, Annual review of clinical psychology.

[24]  Susanne Rässler,et al.  A Non‐Iterative Bayesian Approach to Statistical Matching , 2003 .

[25]  Gracelyn Cruden,et al.  Advancing Science Through Collaborative Data Sharing and Synthesis , 2013, Perspectives on psychological science : a journal of the Association for Psychological Science.

[26]  John B Carlin,et al.  Comparison of methods for imputing limited-range variables: a simulation study , 2014, BMC Medical Research Methodology.

[27]  J. Rosenthal,et al.  Markov Chain Monte Carlo , 2018 .

[28]  Michele L. Ybarra,et al.  Center for Epidemiologic Studies Depression Scale: Review and Revision (CESD and CESD-R). , 2004 .

[29]  M. Kovacs The Children's Depression, Inventory (CDI). , 1985, Psychopharmacology bulletin.

[30]  R. Riley,et al.  Meta-analysis of individual participant data: rationale, conduct, and reporting , 2010, BMJ : British Medical Journal.

[31]  Parminder Raina,et al.  Statistical approaches to harmonize data on cognitive measures in systematic reviews are rarely reported. , 2015, Journal of clinical epidemiology.

[32]  Yulei He,et al.  Diagnosing imputation models by applying target analyses to posterior replicates of completed data , 2012, Statistics in medicine.

[33]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[34]  L. Radloff The use of the Center for Epidemiologic Studies Depression Scale in adolescents and young adults , 1991, Journal of youth and adolescence.

[35]  Roger A. Sugden,et al.  Multiple Imputation for Nonresponse in Surveys , 1988 .

[36]  Xiao-Li Meng,et al.  POSTERIOR PREDICTIVE ASSESSMENT OF MODEL FITNESS VIA REALIZED DISCREPANCIES , 1996 .

[37]  David Kline,et al.  Comparing multiple imputation methods for systematically missing subject‐level data , 2017, Research synthesis methods.

[38]  L. Radloff The CES-D Scale , 1977 .

[39]  Jerome P. Reiter,et al.  Incorporating Marginal Prior Information in Latent Class Models , 2016 .

[40]  A. Gelman,et al.  Not Asked and Not Answered: Multiple Imputation for Multiple Surveys , 1998 .

[41]  Daniel J Bauer,et al.  Psychometric approaches for developing commensurate measures across independent studies: traditional and new models. , 2009, Psychological methods.

[42]  Xiao-Hua Zhou,et al.  Multiple imputation: review of theory, implementation and software , 2007, Statistics in medicine.