Estimation and Uses of the Proportion of Method Variance for Multitrait-Multimethod Data

Recent evidence supports the use of the correlated uniqueness model over the general confirmatory factor analysis model for multitrait-multimethod (MTMM) data The former provides no method factor loadings and therefore no obvious estimate of the proportion of method variance. This is problematic; researchers have used proportions of method variance from the general confirmatory factor analysis model for many purposes. Demonstrations showed that correlated uniquenesses can be averaged to estimate the proportion of method variance. Unlike factor loadings, correlated uniquenesses need not be squared. The first demonstration included a mathematical proof and analysis of an artificial matrix The second involved two employment interview matrixes and one performance appraisal matrix Proportion of method variance estimates were used to make comparisons within and between MTMM matrixes. Results suggested that more structured employment interviews contain smaller proportions of method variance and that peer performance ratings contain greater proportions of method variance than do self-ratings.

[1]  R. Bagozzi Assessing Construct Validity in Personality Research: Applications to Measures of Self-Esteem , 1993 .

[2]  H W Marsh,et al.  Overcoming Problems in Confirmatory Factor Analyses of MTMM Data: The Correlated Uniqueness Model and Factorial Invariance. , 1992, Multivariate behavioral research.

[3]  J. Conway,et al.  Analysis and Design of Multitrait-Multirater Performance Appraisal Studies , 1996 .

[4]  M. Mount,et al.  Psychometric properties of subordinate ratings of managerial performance. , 1984 .

[5]  Larry K. Gaines,et al.  Reliability and validity of the oral interview board in police promotions: A research note , 1982 .

[6]  W. McGaghie,et al.  Construct Validity of Medical Clinical Competence Measures: A Multitrait-Multimethod Matrix Study Using Confirmatory Factor Analysis , 1986 .

[7]  Joseph A. Cote,et al.  Lack of method variance in self-reported affect and perceptions at work: Reality or artifact? , 1989 .

[8]  Joseph A. Cote,et al.  Combining Methodologies in the Construct Validation Process: An Empirical Illustration , 1987 .

[9]  John E. Hunter,et al.  Halo in a multidimensional forced-choice performance evaluation scale. , 1980 .

[10]  H. Marsh Confirmatory Factor Analyses of Multitrait-Multimethod Data: Many Problems and a Few Solutions , 1989 .

[11]  B. B. Saari,et al.  Behavior, Situation, And Rater Variance In Descriptions Of Leader Behaviors. , 1978, Multivariate behavioral research.

[12]  N. Schmitt Method bias: The importance of theory and measurement , 1994 .

[13]  Herbert W. Marsh,et al.  Confirmatory Factor Analysis of Multitrait Multimethod Matrices. , 1983 .

[14]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[15]  D. A. Kenny,et al.  Analysis of the multitrait^multimethod matrix by confirmatory factor analysis , 1992 .

[16]  B. Byrne,et al.  Modeling MTMM Data from Additive and Multiplicative Covariance Structures: An Audit of Construct Validity Concordance. , 1993, Multivariate behavioral research.

[17]  Larry J. Williams,et al.  Measurement and nonmeasurement processes with negative affectivity and employee attitudes , 1996 .

[18]  J. Thayer,et al.  Time urgency: the construct and its measurement. , 1991, The Journal of applied psychology.

[19]  Charles E. Lance,et al.  Specification of the criterion construct space: An application of hierarchical confirmatory factor analysis. , 1992 .

[20]  Joseph A. Cote,et al.  Estimating Trait, Method, and Error Variance: Generalizing across 70 Construct Validation Studies: , 1987 .

[21]  Robert L. Holzbach,et al.  Rater bias in performance ratings: Superior, self-, and peer ratings. , 1978 .

[22]  K. Widaman Hierarchically Nested Covariance Structure Models for Multitrait-Multimethod Data , 1985 .

[23]  H. Marsh,et al.  Confirmatory Factor Analyses of Multitrait-Multimethod Data: A Comparison of Alternative Models , 1991 .

[24]  Michael J. Kavanagh,et al.  Issues in managerial performance: Multitrait-multimethod analyses of ratings. , 1971 .

[25]  Richard P. Bagozzi,et al.  Assessing Construct Validity in Organizational Research , 1991 .

[26]  Michael M. Harris,et al.  Wording Effects in the Measurement of Role Conflict and Role Ambiguity: A Multitrait-Multimethod Analysis , 1994 .

[27]  M. Coovert,et al.  Construct validity of multiple job performance measures using confirmatory factor analysis. , 1988 .