Effects of Stem and Likert Response Option Reversals on Survey Internal Consistency: If You Feel the Need, There is a Better Alternative to Using those Negatively Worded Stems

The controversy with regard to using reverse or negatively worded survey stems has been around for several decades; it is a practice of questionable utility intended to guard against acquiescence or response set behaviors. A 2 × 3 design in which item stem direction and item response pattern direction were crossed was used to determine effects on internal consistency reliability as measured by Cronbach’s alpha. The condition having the highest alpha was when all directly worded stems were used with bidirectional response options. Alpha was higher and accounted for at least 10%, and in one case 20%, higher internal consistency as compared with any of the three conditions in which negatively worded stems were used. This would indicate that the use of all directly worded stems and half of the response options going in one direction and half going in the other direction may be a better way of guarding against acquiescence and response set behaviors than the use of items with negatively worded stems.

[1]  David J. Woodruff,et al.  Statistical Inference for Coefficient Alpha , 1987 .

[2]  P. V. Marsden,et al.  Handbook of Survey Research , 1985 .

[3]  John W. Wick.  Measurement and evaluation in education and psychology. , 1980 .

[4]  L. Cronbach.  Further Evidence on Response Sets and Test Design , 1950 .

[5]  D. Andrich.  Rating Scale Analysis , 1999 .

[6]  Chester A. Schriesheim,et al.  Controlling Acquiescence Response Bias by Item Reversals: The Effect on Questionnaire Validity , 1981 .

[7]  T. Vacha-Haase,et al.  Reliability Generalization: Exploring Variance in Measurement Error Affecting Score Reliability Across Studies , 1998 .

[8]  J. Barnette.  Nonattending Respondent Effects on Internal Consistency of Self-Administered Surveys: A Monte Carlo Simulation Study , 1999 .

[9]  Jeri Benson,et al.  Detecting Item BIAS in Affective Scales , 1987 .

[10]  R. Gable,et al.  The Use of Negative Item Stems: A Cautionary Note. , 1990 .

[11]  Robert K. Gable,et al.  The Impact of Positive and Negative Item Stems on the Validity of a Computer Anxiety Scale , 1990 .

[12]  Jeri Benson,et al.  THE IMPACT OF ITEM PHRASING ON THE VALIDITY OF ATTITUDE SCALES FOR ELEMENTARY SCHOOL CHILDREN , 1985 .

[13]  James D. Wright,et al.  Handbook of Survey Research. , 1985 .

[14]  Chester A. Schriesheim,et al.  The Effect of Negation and Polar Opposite Item Reversals on Questionnaire Reliability and Validity: An Experimental Investigation , 1991 .

[15]  R. Likert.  “Technique for the Measurement of Attitudes, A” , 2022, The SAGE Encyclopedia of Research Design.

[16]  John P. Robinson,et al.  Measures Of Personality And Social Psychological Attitudes , 1991 .

[17]  Jean-Raymond Abrial,et al.  On B , 1998, B.

[18]  R. Sitgreaves.  Psychometric theory (2nd ed.). , 1979 .

[19]  Herbert W. Marsh,et al.  Negative item bias in ratings scales for preadolescent children: a cognitive-developmental phenomenon , 1986 .

[20]  H P Godfrey,et al.  Some normative, reliability, and factor analytic data for the revised UCLA Loneliness Scale. , 1988, Journal of clinical psychology.