Analyzing Multitrait- Multimethod Data A Comparison of Three Approaches

Assessing construct validity is a core task in psychology. Since Campbell and Fiske's (1959) seminal article on multitrait-multimethod (MTMM) analysis, several different methodological approaches for the analysis of convergent and discriminant validity of MTMM data have been developed. In this article, two MTMM approaches are transferred to the general framework of confirmatory factor analysis and compared with the extended version of the correlated trait-correlated method minus one model (Nussbeck, Eid, Geiser, Courvoisier, & Lischetzke, 2009): The multilevel MTMM model (Maas, Lensvelt-Mulders, & Hox, 2009) and the three-mode model (Oort, 2009). Assessing the construct validity of a German Big Five MTMM data set these three MTMM approaches are compared with regard to convergent and discriminant validity estimates and with regard to method effects. Advantages and limitations of each methodological approach will be discussed in detail. Campbell and Fiske's (1959) seminal article on the Conver- gent and discriminant validation by the multitrait- multimethod (MTMM) matrix had an immense impact on the process of exploring construct validity of psychological measures. They proposed to inspect the MTMM matrix, a matrix of correlations between several traits that were mea- sured by several methods, to verify the construct validity of psychological measures. In Campbell and Fiske's perspec- tive the core aspects of construct validity are convergent and discriminant validity. Convergent validity can only be assumed if different methods converge in the measurement of one trait. Discriminant validity, however, should be dem- onstrated by the nonconvergence of measurements of differ- ent traits. The application of at least two different methods for the measurement of one or more traits is necessary to separate trait and method influences for each score of a psy- chological variable (= trait-method unit, TMU). However, within the Campbell and Fiske approach trait- and method-specific influences cannot be separated from unsys- tematic measurement errors. The separation of measurement error from trait- and method-specific influences can be obtained by appropriate modern methodological approaches.

[1]  F. Oort.  Three-Mode Models for Multitrait-Multimethod Data , 2009 .

[2]  Cora J. M. Maas,et al.  A Multilevel Multitrait-Multimethod Analysis , 2009 .

[3]  Michael Eid,et al.  A CTC(M−1) Model for Different Types of Raters , 2009 .

[4]  M. Eid,et al.  Structural equation modeling of multitrait-multimethod data: different models for different types of methods. , 2008, Psychological methods.

[5]  M. Eid,et al.  Analyzing the convergent and discriminant validity of states and traits: development and applications of multimethod latent state-trait models. , 2008, Psychological assessment.

[6]  Michael Eid,et al.  On the meaning of the latent variables in the CT-C(M-1) model: a comment on Maydeu-Olivares and Coffman (2006). , 2008, Psychological methods.

[7]  H. Moosbrugger,et al.  Testtheorie und Fragebogenkonstruktion , 2008 .

[8]  Steffi Pohl,et al.  Modelling method effects as individual causal effects , 2007 .

[9]  J. Hox,et al.  Multilevel Models for Multimethod Measurements. , 2006 .

[10]  Michael Eid,et al.  Methodological Approaches for Analyzing Multimethod Data. , 2006 .

[11]  Jürgen Rost,et al.  Multimethod Item Response Theory. , 2006 .

[12]  Michael Eid,et al.  Introduction: The Need for Multimethod Measurement in Psychology. , 2006 .

[13]  Fridtjof W. Nussbeck.  Assessing Multimethod Association with Categorical Variables , 2006 .

[14]  Michael Eid,et al.  Structural Equation Models for Multitrait-Multimethod Data , 2006 .

[15]  E. Diener,et al.  Handbook of Multimethod Measurement in Psychology , 2005 .

[16]  S. West,et al.  Towards understanding assessments of the big five: multitrait-multimethod analyses of convergent and discriminant validity across measurement occasion and type of observer. , 2004, Journal of personality.

[17]  K. Schermelleh-Engel,et al.  Decomposing person and occasion-specific effects: an extension of latent state-trait (LSI) theory to hierarchical LST models. , 2004, Psychological methods.

[18]  Scott B. MacKenzie,et al.  Common method biases in behavioral research: a critical review of the literature and recommended remedies. , 2003, The Journal of applied psychology.

[19]  M. Eid,et al.  Separating trait effects from trait-specific method effects in multitrait-multimethod models: a multiple-indicator CT-C(M-1) model. , 2003, Psychological methods.

[20]  Timothy J. Robinson,et al.  Multilevel Analysis: Techniques and Applications , 2002 .

[21]  Jürgen Rost,et al.  Multidimensional Rasch Measurement via Item Component Models and Faceted Designs , 2002 .

[22]  Michael Eid,et al.  A multitrait-multimethod model with minimal assumptions , 2000 .

[23]  L. Dumenci,et al.  Multitrait-multimethod Analysis , 2000 .

[24]  Steven D. Brown,et al.  Handbook of applied multivariate statistics and mathematical modeling , 2000 .

[25]  Frans J. Oort,et al.  Stochastic three‐mode models for mean and covariance structures , 1999 .

[26]  Roel Bosker,et al.  Multilevel analysis : an introduction to basic and advanced multilevel modeling , 1999 .

[27]  R. Hoyle.  Structural equation modeling: concepts, issues, and applications , 1997 .

[28]  Herbert W. Marsh,et al.  Latent variable models of multitrait-multimethod data. , 1995 .

[29]  R. Bosker.  Boekbespreking van "A.S. Bryk & S.W. Raudenbusch - Hierarchical linear models: Applications and data analysis methods" : Sage Publications, Newbury Parki, London/New Delhi 1992 , 1995 .

[30]  J. S. Long,et al.  Testing Structural Equation Models , 1993 .

[31]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[32]  H. Marsh.  Confirmatory Factor Analyses of Multitrait-Multimethod Data: Many Problems and a Few Solutions , 1989 .

[33]  H. Marsh,et al.  A new, more powerful approach to multitrait-multimethod analyses: Application of second-order confirmatory factor analysis. , 1988 .

[34]  K. Widaman.  Hierarchically Nested Covariance Structure Models for Multitrait-Multimethod Data , 1985 .

[35]  M. Browne,et al.  The decomposition of multitrait-multimethod matrices. , 1984, The British journal of mathematical and statistical psychology.

[36]  Peter M. Bentler,et al.  A statistical development of three-mode factor analysis , 1979 .

[37]  D. A. Kenny,et al.  An empirical application of confirmatory factor analysis to the multitrait-multimethod matrix , 1976 .

[38]  B. Bloxom.  A note on invariance in three-mode factor analysis , 1968 .

[39]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.