Method Variance From the Perspectives of Reviewers: Poorly Understood Problem or Overemphasized Complaint?

Beliefs and practices regarding common method variance (CMV) were surveyed from a sample of top journal board members. Results indicated that reviewers frequently mentioned CMV concerns and believed that addressing this issue in the design stage was more effective than postdata approaches. Although there was little consensus regarding whether concerns about CMV would cause them to recommend rejection of manuscripts, reviewers generally agreed that some variables were more likely than others to be affected by CMV and that a simple focus on the method of measurement was not adequate for determining whether CMV caused problems with the interpretation of study results. Slight differences were observed across journals and were related to participant experience.

[1]  Joseph A. Cote,et al.  Additive and Multiplicative Method Effects in Applied Psychological Research: An Empirical Assessment of Three Models , 1994 .

[2]  Paul E. Spector.  Method variance as an artifact in self-reported affect and perceptions at work: Myth or significant problem? , 1987 .

[3]  Larry J. Williams,et al.  An Alternative Approach to Method Effects by Using Latent-Variable Models: Applications in Organizational Behavior Research , 1994 .

[4]  William H. Glick,et al.  Common Methods Bias: Does Common Methods Variance Really Bias Results? , 1998 .

[5]  J. George,et al.  The role of negative affectivity in understanding relations between self-reports of stressors and strains: a comment on the applied psychology literature. , 1993, The Journal of applied psychology.

[6]  Larry J. Williams,et al.  Measurement and nonmeasurement processes with negative affectivity and employee attitudes , 1996 .

[7]  T. Judge,et al.  The effect of physical height on workplace success and income: preliminary test of a theoretical model. , 2004, The Journal of applied psychology.

[8]  S. Scullen.  Using Confirmatory Factor Analysis of Correlated Uniquenesses to Estimate Method Variance in Multitrait-Multimethod Matrices , 1999 .

[9]  J. Conway.  Understanding Method Variance in Multitrait-Multirater Performance Appraisal Matrices: Examples Using General Impressions and Interpersonal Affect as Measured Method Factors , 1998 .

[10]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[11]  R. H. Moorman,et al.  A meta‐analytic review and empirical test of the potential confounding effects of social desirability response sets in organizational behaviour research , 1992 .

[12]  Joseph A. Cote,et al.  Lack of method variance in self-reported affect and perceptions at work: Reality or artifact? , 1989 .

[13]  Larry J. Williams,et al.  Method Variance in Organizational Behavior and Human Resources Research: Effects on Correlations, Path Coefficients, and Hypothesis Testing , 1994 .

[14]  S L Golding.  Method Variance, Inadequate Constructs, Or Things That Go Bump In The Night? , 1977, Multivariate behavioral research.

[15]  R. Bagozzi,et al.  Assessing method variance in multitrait-multimethod matrices: The case of self-reported affect and perceptions at work. , 1990 .

[16]  Robert J. Vandenberg,et al.  Introduction: Statistical and Methodological Myths and Urban Legends , 2006 .

[17]  Paul E. Spector,et al.  A longitudinal study of relations between job stressors and job strains while controlling for prior negative affectivity and strains. , 2000, The Journal of applied psychology.

[18]  Scott B. MacKenzie,et al.  Common method biases in behavioral research: a critical review of the literature and recommended remedies. , 2003, The Journal of applied psychology.

[19]  M. Frese,et al.  Longitudinal studies in organizational stress research: a review of the literature with reference to methodological issues. , 1996, Journal of occupational health psychology.

[20]  D. W. Fiske,et al.  Theory and techniques of personality measurement. , 1970, Annual review of psychology.