Investigating the Nature of Method Factors Through Multiple Informants: Evidence for a Specific Factor?

When a self-report instrument includes a balanced number of positively and negatively worded items, factor analysts often use method factors to aid model fitting. The nature of these factors, often referred to as acquiescence, is still debated. Relying upon previous results (Alessandri et al., 2010; DiStefano & Motl, 2006, 2008; Rauch, Schweizer, & Moosbrugger, 2007), we submit that the so-called method factors, instead, represent substantive specific factors. This study investigates the convergence of method effects across different observers. The revised Life Orientation Test (Scheier, Carver, & Bridges, 1994) was administered to a sample of 372 adults (57% females), with 372 acquaintances serving as informants. Results showed that a specific factor was detectable both with self- and other-ratings. A significant correlation across informants provided evidence for the convergence of this specific factor. Construct validity was examined by locating this specific factor within a nomological net of personality variables. Theoretical and practical implications of the findings are discussed.

[1]  José M. Tomás,et al.  Rosenberg's Self-Esteem Scale: Two Factors or Method Effects. , 1999 .

[2]  K. Widaman.  Hierarchically Nested Covariance Structure Models for Multitrait-Multimethod Data , 1985 .

[3]  Michael W. Bridges,et al.  Distinguishing optimism from neuroticism (and trait anxiety, self-mastery, and self-esteem): a reevaluation of the Life Orientation Test. , 1994, Journal of personality and social psychology.

[4]  H W Marsh,et al.  Positive and negative global self-esteem: a substantively meaningful distinction or artifactors? , 1996, Journal of personality and social psychology.

[5]  J. Millham,et al.  Need for social approval: Impression management or self-deception? , 1980 .

[6]  K. Bollen,et al.  Pearson's R and Coarsely Categorized Measures , 1981 .

[7]  Peter M. Bentler,et al.  Practical Issues in Structural Modeling , 1987 .

[8]  R. Sitgreaves.  Psychometric theory (2nd ed.). , 1979 .

[9]  Marco Perugini,et al.  Personality described by adjectives: The generalizability of the Big Five to the Italian lexical context , 1994 .

[10]  Pranjal H. Mehta,et al.  Bridging human and animal research: A comparative approach to studies of personality and health , 2008, Brain, Behavior, and Immunity.

[11]  P. Costa,et al.  The five-factor theory of personality. , 2008 .

[12]  K. Schweizer,et al.  Method effects due to social desirability as a parsimonious explanation of the deviation from unidimensionality in LOT-R scores , 2007 .

[13]  Emin Babakus,et al.  The Sensitivity of Confirmatory Maximum Likelihood Factor Analysis to Violations of Measurement Scale and Distributional Assumptions , 1987 .

[14]  Jodie B. Ullman,et al.  Factor Structure of the Penn State Worry Questionnaire , 2004, Assessment.

[15]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[16]  D. W. Fiske.  Construct Invalidity Comes from Method Effects , 1987 .

[17]  R. Motl,et al.  Validity and factorial invariance of the Social Physique Anxiety Scale. , 2000, Medicine and science in sports and exercise.

[18]  Jason McPherson,et al.  The role of item extremity in the emergence of keying-related factors: an exploration with the life orientation test. , 2005, Psychological methods.

[19]  H. Marsh.  Confirmatory Factor Analyses of Multitrait-Multimethod Data: Many Problems and a Few Solutions , 1989 .

[20]  W. R. Dillon,et al.  Inferring Latent Brand Dependencies , 2000 .

[21]  Stephen G. West,et al.  Structural equation models with non-normal variables: Problems and remedies , 1995 .

[22]  Albert Satorra,et al.  A scaled difference chi-square test statistic for moment structure analysis , 1999 .

[23]  Evan F. Risko,et al.  Correlates of the Rosenberg Self-Esteem Scale Method Effects , 2006 .

[24]  A. Mehrabian,et al.  Dimensionality and content of optimism-pessimism analyzed in terms of the pad temperament model , 1997 .

[25]  Shelley E. Taylor,et al.  Psychological resources, positive illusions, and health. , 2000, The American psychologist.

[26]  R. Motl,et al.  The Social Physique Anxiety Scale: an example of the potential consequence of negatively worded items in factorial validity studies. , 2000, Journal of applied measurement.

[27]  O. John,et al.  Accuracy and bias in self-perception: individual differences in self-enhancement and the role of narcissism. , 1994, Journal of personality and social psychology.

[28]  Joseph A. Cote,et al.  Lack of method variance in self-reported affect and perceptions at work: Reality or artifact? , 1989 .

[29]  Delroy L. Paulhus,et al.  Egoistic and Moralistic Biases in Self-Perception: The Interplay of Self-Deceptive Styles With Basic Traits and Motives , 1998 .

[30]  B. de Raad,et al.  The substantive nature of psycholexical personality factors: a comparison across languages. , 2002, Journal of personality and social psychology.

[31]  R. Motl,et al.  Personality correlates of method effects due to negatively worded items on the Rosenberg Self-Esteem scale , 2009 .

[32]  Scott B. MacKenzie,et al.  Common method biases in behavioral research: a critical review of the literature and recommended remedies. , 2003, The Journal of applied psychology.

[33]  Jaak Billiet,et al.  Modeling Acquiescence in Measurement Models for Two Balanced Sets of Items , 2000 .

[34]  H. Marsh.  Multitrait-Multimethod Analyses: Inferring Each Trait-Method Combination with Multiple Indicators. , 1993 .

[35]  P. Bentler,et al.  Much More Than Model Fitting? Evidence for the Heritability of Method Effect Associated With Positively Worded Items of the Life Orientation Test Revised , 2010 .

[36]  Christine DiStefano,et al.  Wording Effects in Self-Esteem Scales: Methodological Artifact or Response Style? , 2003 .

[37]  Lewis R Goldberg,et al.  A hierarchical analysis of 1,710 English personality-descriptive adjectives. , 2004, Journal of personality and social psychology.

[38]  Christine DiStefano,et al.  Further Investigating Method Effects Associated With Negatively Worded Items on Self-Report Surveys , 2006 .

[39]  B. Muthén,et al.  A comparison of some methodologies for the factor analysis of non‐normal Likert variables , 1985 .

[40]  P. Bentler,et al.  Fit indices in covariance structure modeling : Sensitivity to underparameterized model misspecification , 1998 .

[41]  Paul E. Spector.  A Consideration of the Validity and Meaning of Self-Report Measures of Job Conditions , 1992 .

[42]  Richard P. Bagozzi,et al.  An examination of the psychometric properties of measures of negative affect in the PANAS-X scales. , 1993 .

[43]  E. Raufaste,et al.  Dimensionality of the Revised Life Orientation Test and the status of filler items , 2003 .

[44]  S. West,et al.  Towards understanding assessments of the big five: multitrait-multimethod analyses of convergent and discriminant validity across measurement occasion and type of observer. , 2004, Journal of personality.

[45]  R. Corwyn,et al.  Mean And Covariance Structures Analyses: An Examination Of The Rosenberg Self-Esteem Scale Among Adolescents And Adults , 2003 .

[46]  L. Cronbach.  Response Sets and Test Validity , 1946 .

[47]  Herbert W. Marsh,et al.  Positive and negative global self-esteem: a substantively meaningful distinction or artifactors? , 1996 .

[48]  R. Motl,et al.  Longitudinal Invariance of Self-Esteem and Method Effects Associated With Negatively Worded Items , 2002 .

[49]  D. Paulhus.  Measurement and control of response bias. , 1991 .

[50]  Edward G. Carmines,et al.  Reliability and Validity Assessment , 1979 .

[51]  B. Muthén,et al.  A comparison of some methodologies for the factor analysis of non‐normal Likert variables: A note on the size of the model , 1992 .

[52]  Jürgen Hoyer,et al.  Separating Optimism and Pessimism: a Robust Psychometric Analysis of the Revised Life Orientation Test (lot–r) the Internal Structure of the Revised Life Orientation Test (lot , 2022 .

[53]  D. A. Kenny,et al.  Analysis of the multitrait^multimethod matrix by confirmatory factor analysis , 1992 .

[54]  L. Kubzansky,et al.  Optimism and Pessimism in the Context of Health: Bipolar Opposites or Separate Constructs? , 2004, Personality & social psychology bulletin.