Psychometric approaches for developing commensurate measures across independent studies: traditional and new models.

When conducting an integrative analysis of data obtained from multiple independent studies, a fundamental problem is to establish commensurate measures for the constructs of interest. Fortunately, procedures for evaluating and establishing measurement equivalence across samples are well developed for the linear factor model and commonly used item response theory models. A newly proposed moderated nonlinear factor analysis model generalizes these models and procedures, allowing for items of different scale types (continuous or discrete) and differential item functioning across levels of categorical and/or continuous variables. The potential of this new model to resolve the problem of measurement in integrative data analysis is shown via an empirical example examining changes in alcohol involvement from ages 10 to 22 years across 2 longitudinal studies.

[1]  Jan de Leeuw,et al.  On the relationship between item response theory and factor analysis of discretized variables , 1987 .

[2]  David Thissen,et al.  Item Response Theory for Items Scored in Two Categories , 2001 .

[3]  Akihito Kamata,et al.  A Note on the Relation Between Factor Analytic and Item Response Theory Models , 2008 .

[4]  Keith F Widaman,et al.  Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance. , 1993, Psychological bulletin.

[5]  W. H. Angoff,et al.  Perspectives on differential item functioning methodology. , 1993 .

[6]  P. Boeck,et al.  Explanatory item response models : a generalized linear and nonlinear approach , 2004 .

[7]  C. Edelbrock,et al.  The classification of child psychopathology: a review and analysis of empirical efforts. , 1978, Psychological bulletin.

[8]  Michael C Neale,et al.  People are variables too: multilevel structural equations modeling. , 2005, Psychological methods.

[9]  K. Sher,et al.  Characteristics of children of alcoholics: putative risk factors, substance use and abuse, and psychopathology. , 1991, Journal of abnormal psychology.

[10]  P. Fayers.  Item Response Theory for Psychologists , 2004, Quality of Life Research.

[11]  Stan Lipovetsky,et al.  Generalized Latent Variable Modeling: Multilevel,Longitudinal, and Structural Equation Models , 2005, Technometrics.

[12]  Pickles A SkrondalA. Rabe-HeskethS.  GLLAMM: A general class of multilevel models and a STATA programme , 2001 .

[13]  Gordon W. Cheung,et al.  Testing Factorial Invariance across Groups: A Reconceptualization and Proposed New Method , 1999 .

[14]  Geert Molenberghs,et al.  An Introduction to (Generalized (Non)Linear Mixed Models , 2004 .

[15]  W. Meredith,et al.  Factorial Invariance: Historical Perspectives and New Problems , 2007 .

[16]  Thomas R. Boucher,et al.  Test Equating, Scaling, and Linking: Methods and Practices , 2007 .

[17]  Kristopher J Preacher,et al.  Item factor analysis: current approaches and future directions. , 2007, Psychological methods.

[18]  D. Borsboom.  Educational Measurement (4th ed.) , 2009 .

[19]  A. Meade,et al.  Examining Question and Context Effects in Organization Survey Data Using Item Response Theory , 2009 .

[20]  B. French,et al.  Multigroup Confirmatory Factor Analysis: Locating the Invariant Referent Sets , 2008 .

[21]  D. Flora,et al.  Defining risk heterogeneity for internalizing symptoms among children of alcoholic parents , 2008, Development and Psychopathology.

[22]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters , 1982 .

[23]  B. Muthén.  Latent variable modeling in heterogeneous populations , 1989 .

[24]  L. Chassin,et al.  Externalizing symptoms among children of alcoholic parents: Entry points for an antisocial pathway to alcoholism. , 2007, Journal of abnormal psychology.

[25]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[26]  Patricio Cumsille,et al.  Second-order latent growth models. , 2001 .

[27]  Paul F. Lazarsfeld,et al.  Latent Structure Analysis. , 1969 .

[28]  Li Cai.  A Metropolis-Hastings Robbins-Monro Algorithm for Maximum Likelihood Nonlinear Latent Structure Analysis with a Comprehensive Measurement Model , 2008 .

[29]  J. Schafer,et al.  Missing data: our view of the state of the art. , 2002, Psychological methods.

[30]  D. Sörbom.  A GENERAL METHOD FOR STUDYING DIFFERENCES IN FACTOR MEANS AND FACTOR STRUCTURE BETWEEN GROUPS , 1974 .

[31]  Howard Wainer,et al.  Detection of differential item functioning using the parameters of item response models. , 1993 .

[32]  W. Wothke.  Longitudinal and multigroup modeling with missing data. , 2000 .

[33]  M. Browne.  Asymptotically distribution-free methods for the analysis of covariance structures. , 1984, The British journal of mathematical and statistical psychology.

[34]  S. Reise,et al.  Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. , 1997 .

[35]  M. Barrera,et al.  Substance use and symptomatology among adolescent children of alcoholics. , 1991, Journal of abnormal psychology.

[36]  W. Meredith.  Measurement invariance, factor analysis and factorial invariance , 1993 .

[37]  John A. Nelder,et al.  Generalized linear models. 2nd ed. , 1993 .

[38]  B. Byrne,et al.  Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. , 1989 .

[39]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[40]  J-P Fox.  Stochastic EM for estimating the parameters of a multilevel IRT model. , 2003, The British journal of mathematical and statistical psychology.

[41]  Linda M. Collins,et al.  New methods for the analysis of change , 2001 .

[42]  Neil Henry.  Latent structure analysis , 1969 .

[43]  G. A. Marcoulides,et al.  Advanced structural equation modeling : issues and techniques , 1996 .

[44]  J. Fox,et al.  Bayesian estimation of a multilevel IRT model using gibbs sampling , 2001 .

[45]  P. McCullagh,et al.  Generalized Linear Models, 2nd Edn. , 1990 .

[46]  Marcello D'Orazio,et al.  Statistical Matching: Theory and Practice , 2006 .

[47]  A. Alas,et al.  BAYESIAN ESTIMATION OF A MULTILEVEL IRT MODEL USING GIBBS SAMPLING JEAN-PAUL FOX AND CEES , 2005 .

[48]  James L. Arbuckle,et al.  Full Information Estimation in the Presence of Incomplete Data , 1996 .

[49]  Robert Cudeck,et al.  Factor analysis at 100 : historical developments and future directions , 2007 .

[50]  J W Grice,et al.  Computing and Evaluating Factor Scores , 2004 .

[51]  James Lani,et al.  Brief Symptom Inventory (BSI) , 2010 .

[52]  Jürgen Baumert,et al.  Modeling longitudinal and multilevel data , 2000 .

[53]  Paul W. Holland,et al.  A Framework and History for Score Linking , 2007 .

[54]  Harvey Goldstein,et al.  MULTILEVEL MODELLING NEWSLETTER , 2002 .

[55]  R. Linn.  Educational measurement, 3rd ed. , 1989 .

[56]  Wenjing Huang,et al.  Pooling data from multiple longitudinal studies: the role of item response theory in integrative data analysis. , 2008, Developmental psychology.

[57]  Daniel J. Bauer.  Estimating Multilevel Linear Models as Structural Equation Models , 2003 .

[58]  Michael Windle,et al.  The science of prevention: methodological advances from alcohol and substance abuse research. , 1997 .

[59]  Tammy Chung,et al.  Item response theory analysis of diagnostic criteria for alcohol and cannabis use disorders in adolescents: implications for DSM-V. , 2006, Journal of abnormal psychology.

[60]  Andrea M Hussong,et al.  Integrative data analysis: the simultaneous analysis of multiple data sets. , 2009, Psychological methods.

[61]  R. Brennan,et al.  Test Equating, Scaling, and Linking: Methods and Practices , 2004 .

[62]  P. Holland,et al.  Linking and aligning scores and scales , 2007 .

[63]  K. G. J8reskoC,et al.  Simultaneous Factor Analysis in Several Populations , 2007 .

[64]  F. Krauss.  Latent Structure Analysis , 1980 .

[65]  David Thissen,et al.  Quick and Easy Implementation of the Benjamini-Hochberg Procedure for Controlling the False Positive Rate in Multiple Comparisons , 2002 .

[66]  Francis Tuerlinckx,et al.  A nonlinear mixed model framework for item response theory. , 2003, Psychological methods.

[67]  J. Steenkamp,et al.  Assessing Measurement Invariance in Cross-National Consumer Research , 1998 .

[68]  Roger E. Millsap,et al.  Assessing Factorial Invariance in Ordered-Categorical Measures , 2004 .

[69]  H. Wainer,et al.  Differential Item Functioning. , 1994 .

[70]  Kristopher J Preacher,et al.  On the practice of dichotomization of quantitative variables. , 2002, Psychological methods.

[71]  R. Millsap.  Four Unresolved Problems in Studies of Factorial Invariance. , 2005 .

[72]  S. Maisto,et al.  Patterns of DSM-IV alcohol abuse and dependence symptoms in adolescent drinkers. , 1995, Journal of studies on alcohol.

[73]  I. Moustaki.  A latent trait and a latent class model for mixed observed variables , 1996 .

[74]  Roger E. Millsap,et al.  Detecting Violations of Factorial Invariance Using Data-Based Specification Searches: A Monte Carlo Study , 2007 .

[75]  Cees A. W. Glas,et al.  DETECTION OF DIFFERENTIAL ITEM FUNCTIONING USING LAGRANGE MULTIPLIER TESTS , 1996 .

[76]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm , 1981 .

[77]  Stan Lipovetsky,et al.  Latent Variable Models and Factor Analysis , 2001, Technometrics.

[78]  Patrick J Curran,et al.  Have Multilevel Models Been Structural Equation Models All Along? , 2003, Multivariate behavioral research.