A Comparison of Method Effects in Two Confirmatory Factor Models for Structurally Different Methods

Multimethod data analysis is a complex procedure that is often used to examine the degree to which different measures of the same construct converge in the assessment of this construct. Several authors have called for a greater understanding of the definition and meaning of method effects in different models for multimethod data. In this article, we compare 2 recently proposed approaches for modeling data with structurally different methods with regard to the definition and meaning of method effects, the restricted CT-C(M – 1) model (Geiser, Eid, & Nussbeck, 2008) and the latent difference model (Lischetzke, Eid, & Nussbeck, 2002). We also introduce the concepts of individual, conditional, and general method bias and show how these types of biases are represented in the models. An application to a multirater data set (N = 199) as well as recommendations for the application and interpretation of each model are provided.

[1]  R. Millsap.  The Statistical Analysis of Method Effects in Multitrait-Multimethod Data: A Review , 2014 .

[2]  Herbert W. Marsh,et al.  A New , More Powerful Approach to Multitrait-Multimethod Analyses : An Application of Second-order Confirmatory Factor Analysis , 2011 .

[3]  E. Grigorenko,et al.  Cross-informant symptoms from CBCL, TRF, and YSR: trait and method variance in a normative sample of Russian youths. , 2010, Psychological assessment.

[4]  Steffi Pohl,et al.  Modeling Common Traits and Method Effects in Multitrait-Multimethod Analysis , 2010, Multivariate behavioral research.

[5]  T. Brown,et al.  The construct validity of acceptance: a multitrait-multimethod investigation. , 2009, Behavior therapy.

[6]  Michael Eid,et al.  A CTC(M−1) Model for Different Types of Raters , 2009 .

[7]  S. Vautier,et al.  Do balanced scales assess bipolar constructs? The case of the STAI scales. , 2009, Psychological assessment.

[8]  N. Kämpfe,et al.  Personality and the four faces of positive affect: A multitrait-multimethod analysis using self- and peer-report , 2008 .

[9]  M. Eid,et al.  Structural equation modeling of multitrait-multimethod data: different models for different types of methods. , 2008, Psychological methods.

[10]  Michael Eid,et al.  On the meaning of the latent variables in the CT-C(M-1) model: a comment on Maydeu-Olivares and Coffman (2006). , 2008, Psychological methods.

[11]  Steffi Pohl,et al.  Modelling method effects as individual causal effects , 2007 .

[12]  S. Vautier,et al.  Analyzing Individual Differences in Vocational, Leisure, and Family Interests: A Multitrait-Multimethod Approach. , 2007 .

[13]  R. Devellis.  Classical Test Theory , 2006, Medical care.

[14]  M. Pero-Cebollero,et al.  DSM-IV Attention Deficit Hyperactivity Disorder Symptoms: Agreement Between Informants in Prevalence and Factor Structure at Different Ages , 2006 .

[15]  Michael Eid,et al.  Structural Equation Models for Multitrait-Multimethod Data , 2006 .

[16]  E. Diener,et al.  Handbook of Multimethod Measurement in Psychology , 2005 .

[17]  M. Eid,et al.  Separating trait effects from trait-specific method effects in multitrait-multimethod models: a multiple-indicator CT-C(M-1) model. , 2003, Psychological methods.

[18]  M. Eid,et al.  The structure of emotional expressivity: each emotion counts. , 2002, Journal of personality and social psychology.

[19]  D. Bandalos.  The Effects of Item Parceling on Goodness-of-Fit and Parameter Estimate Bias in Structural Equation Modeling , 2002 .

[20]  J. Mcardle,et al.  Latent difference score structural models for linear dynamic analyses with incomplete longitudinal data. , 2001 .

[21]  Michael Eid,et al.  A multitrait-multimethod model with minimal assumptions , 2000 .

[22]  L. Dumenci,et al.  Multitrait-multimethod Analysis , 2000 .

[23]  Steven D. Brown,et al.  Handbook of applied multivariate statistics and mathematical modeling , 2000 .

[24]  Michael Eid,et al.  Longitudinal Con rmatory Factor Analysis for Polytomous Item Responses: Model De nition and Model Selection on the Basis of Stochastic Measurement Theory , 1998 .

[25]  M. Eid,et al.  Modeling true intraindividual change: True change as a latent variable. , 1997 .

[26]  Lee J. Cronbach,et al.  Giving method variance its due. , 1995 .

[27]  Werner Wothke,et al.  Covariance components analysis of the multitrait-multimethod matrix. , 1995 .

[28]  Donald T. Campbell,et al.  Citations do not solve problems. , 1992 .

[29]  H. Marsh,et al.  Confirmatory Factor Analyses of Multitrait-Multimethod Data: A Comparison of Alternative Models , 1991 .

[30]  Rolf Steyer,et al.  Models of classical psychometric test theory as stochastic measurement models: Representation, uniqueness, meaningfulness, identifiability, and testability. , 1989 .

[31]  D. Ozer.  Construct Validity in Personality Assessment , 1989 .

[32]  P. Holland.  Statistics and Causal Inference , 1985 .

[33]  K. Widaman.  Hierarchically Nested Covariance Structure Models for Multitrait-Multimethod Data , 1985 .

[34]  J. Deegan.  On the Occurrence of Standardized Regression Coefficients Greater Than One , 1978 .

[35]  D. A. Kenny,et al.  An empirical application of confirmatory factor analysis to the multitrait-multimethod matrix , 1976 .

[36]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[37]  Donald W. Zimmerman.  Probability spaces, hilbert spaces, and the axioms of test theory , 1975 .

[38]  K. G. Jöreskog,et al.  Statistical analysis of sets of congeneric tests , 1971 .

[39]  M. R. Novick.  The axioms and principal results of classical test theory , 1965 .

[40]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.