Further Investigating Method Effects Associated With Negatively Worded Items on Self-Report Surveys

This article used multitrait-multimethod methodology and covariance modeling for an investigation of the presence and correlates of method effects associated with negatively worded items on the Rosenberg Self-Esteem (RSE) scale (Rosenberg, 1989) using a sample of 757 adults. Results showed that method effects associated with negative item phrasing on the RSE scale were present. Method effects associated with negative item wording were similarly observed with the Social Physique Anxiety Scale (SPAS; Hart, Leary, & Rejeski, 1989) and method effects were present and significantly correlated in analyses that included both the RSE scale and SPAS simultaneously. Path analysis modeling that incorporated personality measures identified factors that correlated with the presence of method effects. These findings further suggest that method effects associated with negatively worded items may be considered a response style.

[1]  Edward G. Carmines,et al.  Reliability and Validity Assessment , 1979 .

[2]  P. O'Malley,et al.  Black-White Differences in Self-Esteem: Are They Affected by Response Styles? , 1984, American Journal of Sociology.

[3]  Sewall Wright,et al.  Path coefficients and path regressions: Alternative or complementary concepts? , 1960 .

[4]  P M Bentler,et al.  Identification of content and style: a two-dimensional interpretation of acquiescence. , 1971, Psychological bulletin.

[5]  Christine DiStefano,et al.  Wording Effects in Self-Esteem Scales: Methodological Artifact or Response Style? , 2003 .

[6]  G. Marín,et al.  Extreme Response Style and Acquiescence among Hispanics , 1992 .

[7]  Herbert W. Marsh,et al.  Latent variable models of multitrait-multimethod data. , 1995 .

[8]  David E. Conroy,et al.  Progress in the development of a multidimensional measure of fear of failure: The performance failure appraisal inventory (pfai) , 2001 .

[9]  R. Motl,et al.  The Social Physique Anxiety Scale: an example of the potential consequence of negatively worded items in factorial validity studies. , 2000, Journal of applied measurement.

[10]  M. Leary.  A Brief Version of the Fear of Negative Evaluation Scale , 1983 .

[11]  Jaak Billiet,et al.  Modeling Acquiescence in Measurement Models for Two Balanced Sets of Items , 2000 .

[12]  Edward McAuley,et al.  The Social Physique Anxiety Scale: construct validity in adolescent females. , 1993, Medicine and science in sports and exercise.

[13]  D. Jackson,et al.  Content and style in personality assessment. , 2011, Psychological bulletin.

[14]  K. Rasinski,et al.  Quality of the Responses of Eighth-Grade Students in NELS:88--National Education Longitudinal Study of 1988. Technical Report. , 1991 .

[15]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[16]  E. Greenleaf,et al.  MEASURING EXTREME RESPONSE STYLE , 1992 .

[17]  H W Marsh,et al.  Overcoming Problems in Confirmatory Factor Analyses of MTMM Data: The Correlated Uniqueness Model and Factorial Invariance. , 1992, Multivariate behavioral research.

[18]  L. Cronbach.  Further Evidence on Response Sets and Test Design , 1950 .

[19]  Robert J. Vandenberg,et al.  A Central Question in Cross-Cultural Research: Do Employees of Different Cultures Interpret Work-related Measures in an Equivalent Manner?: , 1994 .

[20]  M. Leary,et al.  The Measurement of Social Physique Anxiety , 1989 .

[21]  M. Rosenberg.  Society and the Adolescent Self-Image. Rev. ed. , 1989 .

[22]  R. Davidson.  Affective Style and Affective Disorders: Perspectives from Affective Neuroscience , 1998 .

[23]  Karl G. Jöreskog,et al.  Lisrel 8: User's Reference Guide , 1997 .

[24]  Herbert W. Marsh,et al.  Negative item bias in ratings scales for preadolescent children: a cognitive-developmental phenomenon , 1986 .

[25]  S. Mulaik,et al.  EVALUATION OF GOODNESS-OF-FIT INDICES FOR STRUCTURAL EQUATION MODELS , 1989 .

[26]  C. Carver,et al.  Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The BIS/BAS Scales , 1994 .

[27]  M. Scheier,et al.  Public and private self-consciousness: Assessment and theory. , 1975 .

[28]  M. Kohn,et al.  Class and conformity : a study in values , 1970 .

[29]  Paul E. Spector.  Summated rating scale construction , 1991 .

[30]  H. Marsh.  Confirmatory Factor Analyses of Multitrait-Multimethod Data: Many Problems and a Few Solutions , 1989 .

[31]  R. Bagozzi.  Assessing Construct Validity in Personality Research: Applications to Measures of Self-Esteem , 1993 .

[32]  Irvine Clarke,et al.  Extreme response style in cross-cultural research: An empirical investigation. , 2000 .

[33]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[34]  James C. Anderson,et al.  Monte Carlo Evaluations of Goodness of Fit Indices for Structural Equation Models , 1992 .

[35]  H. J. Greenwald,et al.  A Short Social Desirability Scale , 1970 .

[36]  D. Marlowe,et al.  A new scale of social desirability independent of psychopathology. , 1960, Journal of consulting psychology.

[37]  Self-motivation and physical activity among black and white adolescent girls. , 2003, Medicine and science in sports and exercise.

[38]  R. Motl,et al.  Longitudinal Invariance of Self-Esteem and Method Effects Associated With Negatively Worded Items , 2002 .

[39]  José M. Tomás,et al.  Rosenberg's Self-Esteem Scale: Two Factors or Method Effects. , 1999 .

[40]  Michael W. Bridges,et al.  Distinguishing optimism from neuroticism (and trait anxiety, self-mastery, and self-esteem): a reevaluation of the Life Orientation Test. , 1994, Journal of personality and social psychology.

[41]  H W Marsh,et al.  Positive and negative global self-esteem: a substantively meaningful distinction or artifactors? , 1996, Journal of personality and social psychology.

[42]  Robert F. DeVellis,et al.  Scale Development: Theory and Applications. , 1992 .

[43]  W. E. Hensley,et al.  Dimensions of Rosenburg's Self-Esteem Scale , 1976, Psychological reports.

[44]  Karl G. Jöreskog,et al.  Analyzing psychological data by structural analysis of covariance matrices , 1974 .