Extensions of Multiple-Group Item Response Theory Alignment: Application to Psychiatric Phenotypes in an International Genomics Consortium

Large-scale studies spanning diverse project sites, populations, languages, and measurements are increasingly important to relate psychological to biological variables. National and international consortia already are collecting and executing mega-analyses on aggregated data from individuals, with different measures on each person. In this research, we show that Asparouhov and Muthén’s alignment method can be adapted to align data from disparate item sets and response formats. We argue that with these adaptations, the alignment method is well suited for combining data across multiple sites even when they use different measurement instruments. The approach is illustrated using data from the Whole Genome Sequencing in Psychiatric Disorders consortium and a real-data-based simulation is used to verify accurate parameter recovery. Factor alignment appears to increase precision of measurement and validity of scores with respect to external criteria. The resulting parameter estimates may further inform development of more effective and efficient methods to assess the same constructs in prospectively designed studies.

[1]  David Kaplan,et al.  Data fusion with international large scale assessments: a case study using the OECD PISA and TALIS surveys , 2013 .

[2]  L. Chassin,et al.  Parent Alcoholism Impacts the Severity and Timing of Children’s Externalizing Symptoms , 2010, Journal of abnormal child psychology.

[3]  H. Marsh.  Confirmatory Factor Analyses of Multitrait-Multimethod Data: Many Problems and a Few Solutions , 1989 .

[4]  Andrea M Hussong,et al.  Integrative data analysis: the simultaneous analysis of multiple data sets. , 2009, Psychological methods.

[5]  Dena A. Pastor,et al.  The Use of Multilevel Item Response Theory Modeling in Applied Research: An Illustration , 2003 .

[6]  J. Nurnberger,et al.  Diagnostic interview for genetic studies. Rationale, unique features, and training. NIMH Genetics Initiative. , 1994, Archives of general psychiatry.

[7]  M. Preisig,et al.  Assessment and characterization of phenotypic heterogeneity of anxiety disorders across five large cohorts , 2016, International journal of methods in psychiatric research.

[8]  D. Sheehan,et al.  The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. , 1998, The Journal of clinical psychiatry.

[9]  N C Andreasen,et al.  The Comprehensive Assessment of Symptoms and History (CASH). An instrument for assessing diagnosis and psychopathology. , 1992, Archives of general psychiatry.

[10]  Klaus J. Templer,et al.  Cultural Intelligence: Its Measurement and Effects on Cultural Judgment and Decision Making, Cultural Adaptation and Task Performance , 2007, Management and Organization Review.

[11]  Walter P. Vispoel,et al.  Psychometric properties for the Balanced Inventory of Desirable Responding: dichotomous versus polytomous conventional and IRT scoring. , 2014, Psychological assessment.

[12]  M. de Andrade,et al.  Using Item Response Theory to Model Multiple Phenotypes and Their Joint Heritability in Family Data , 2014, Genetic epidemiology.

[13]  F. Samejima.  Graded Response Model , 1997 .

[14]  D. Betsy McCoach,et al.  An Investigation of the Alignment Method With Polytomous Indicators Under Conditions of Partial Measurement Invariance , 2018 .

[15]  T. A. Warm.  Weighted likelihood estimation of ability in item response theory , 1989 .

[16]  Bengt Muthén,et al.  Recent Methods for the Study of Measurement Invariance With Many Groups , 2018 .

[17]  Chenyang Gu,et al.  Combining item response theory with multiple imputation to equate health assessment questionnaires , 2017, Biometrics.

[18]  Bengt Muthén,et al.  Multiple-Group Factor Analysis Alignment , 2014 .

[19]  Kevin J. Grimm,et al.  Modeling life-span growth curves of cognition using longitudinal data with multiple samples and changing scales of measurement. , 2009, Psychological methods.

[20]  J. Horn,et al.  A contemporary method for developmental‐genetic analyses of age changes in intellectual abilities , 1998 .

[21]  Robert J. Mislevy,et al.  Estimating Population Characteristics From Sparse Matrix Samples of Item Responses , 1992 .

[22]  Robert J. Mislevy,et al.  Randomization-based inference about latent variables from complex samples , 1991 .

[23]  S. Reise,et al.  Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. , 1997 .

[24]  T. Lehner,et al.  Genomic resources for the study of neuropsychiatric disorders , 2017, Molecular Psychiatry.

[25]  B. Avolio,et al.  Context and leadership: An examination of the nine-factor full-range leadership theory using the Multifactor Leadership Questionnaire. , 2003 .

[26]  R. Philip Chalmers,et al.  mirt: A Multidimensional Item Response Theory Package for the R Environment , 2012 .

[27]  Li Cai,et al.  Generalized full-information item bifactor analysis. , 2011, Psychological methods.

[28]  Kevin J Grimm,et al.  Data Integration Approaches to Longitudinal Growth Modeling , 2017, Educational and psychological measurement.

[29]  Bernard P. Veldkamp,et al.  Optimizing Balanced Incomplete Block Designs for Educational Assessments , 2004 .

[30]  B. Muthén,et al.  What to do When Scalar Invariance Fails: The Extended Alignment Method for Multi-Group Factor Analysis Comparison of Latent Means Across Many Groups , 2017, Psychological methods.

[31]  Jan de Leeuw,et al.  On the relationship between item response theory and factor analysis of discretized variables , 1987 .

[32]  P. Holland,et al.  Linking and aligning scores and scales , 2007 .

[33]  R. Vandenberg,et al.  A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research , 2000 .

[34]  Hailiang Huang,et al.  Whole genome sequencing in psychiatric disorders: the WGSPD consortium , 2017, bioRxiv.

[35]  Margaret Wu.  The Role of Plausible Values in Large-Scale Surveys. , 2005 .

[36]  R. Kotov,et al.  Symptoms of psychosis in schizophrenia, schizoaffective disorder, and bipolar disorder: A comparison of African Americans and Caucasians in the Genomic Psychiatry Cohort , 2016, American journal of medical genetics. Part B, Neuropsychiatric genetics : the official publication of the International Society of Psychiatric Genetics.

[37]  Bengt Muthén,et al.  IRT studies of many groups: the alignment method , 2014, Front. Psychol..

[38]  R. Brennan,et al.  Test Equating, Scaling, and Linking: Methods and Practices , 2004 .

[39]  Donald Hedeker,et al.  Full-Information Item Bifactor Analysis of Graded Response Data , 2007 .

[40]  M. First,et al.  Validation of the NetSCID: an automated web-based adaptive version of the SCID. , 2016, Comprehensive psychiatry.

[41]  Akihito Kamata,et al.  A Note on the Relation Between Factor Analytic and Item Response Theory Models , 2008 .

[42]  Raymond J. Adams,et al.  Multilevel Item Response Models: An Approach to Errors in Variables Regression , 1997 .

[43]  D. Flora,et al.  Disaggregating the Distal, Proximal, and Time-Varying Effects of Parent Alcoholism on Children’s Internalizing Symptoms , 2008, Journal of abnormal child psychology.

[44]  D. Rubin.  INFERENCE AND MISSING DATA , 1975 .

[45]  S. Djurovic,et al.  Polygenic dissection of diagnosis and clinical dimensions of bipolar disorder and schizophrenia , 2013, Molecular Psychiatry.

[46]  Scott M Hofer,et al.  Integrative data analysis through coordination of measurement and analysis protocol across independent longitudinal studies. , 2009, Psychological methods.

[47]  S. Schwartz,et al.  Sex differences in value priorities: cross-cultural and multimethod studies. , 2005, Journal of personality and social psychology.

[48]  Dennis A. Revicki,et al.  Handbook of Item Response Theory Modeling : Applications to Typical Performance Assessment , 2014 .

[49]  Walter L Leite,et al.  Item Selection for the Development of Short Forms of Scales Using an Ant Colony Optimization Algorithm , 2008, Multivariate behavioral research.