Middle Alternatives, Acquiescence, and the Quality of Questionnaire Data

Some scholars have suggested that offering a middle alternative on a rating scale is necessary to measure opinions accurately, whereas other scholars have suggested that middle alternatives offer "easy outs" to respondents who want to avoid taking sides on an issue. In this paper, we evaluate these competing hypotheses using data from an experiment conducted in the 1992 Euro-Barometer Survey. Via structural equation modeling of responses to agree/disagree items measuring attitudes toward science and technology, we found that offering a middle alternative reduces the amount of random measurement error in the responses, thereby increasing reliability, while not affecting the validity of attitude measurements. This suggests that middle alternatives should be included in rating scales in order to maximize data quality. We also found evidence of acquiescence response bias in answers to the agree/disagree items; while unrelated to the presence of a middle alternative, this bias was stronger among older, less educated, and female respondents. And controlling for this bias greatly improved the apparent validity of attitude items.

[1]  J. R. Wright,et al.  Perceptions of candidates' issue positions , 1983 .

[2]  R. Tourangeau,et al.  Cognitive Processes Underlying Context Effects in Attitude Measurement , 1988 .

[3]  R. Wyer,et al.  Effects of rank ordering stimuli on magnitude ratings of these and other stimuli , 1985 .

[4]  G. Jenkins,et al.  A Monte Carlo study of factors affecting three indices of composite scale reliability. , 1977 .

[5]  G. Wolkon,et al.  A Forced Choice Form of the F Scale-Free of Acquiescent Response Set , 1964 .

[6]  A. Bachu.  Current Population Reports , 1999 .

[7]  Es R. Masters.  THE RELATIONSHIP BETWEEN NUMBER OF RESPONSE CATEGORIES AND RELIABILITY OF LIKERT‐TYPE QUESTIONNAIRES1 , 1974 .

[8]  James O. Ramsay,et al.  The effect of number of categories in rating scales on precision of estimation of scale values , 1973 .

[9]  How interviewer effects operate through question form. , 1949 .

[10]  Measuring Change in Personal Economic Well-Being , 1986 .

[11]  D. Jackson,et al.  HOMOGENEOUS DIMENSIONS OF PERSONALITY SCALE CONTENT. , 1968, Multivariate behavioral research.

[12]  B. Bass.  Famous Sayings Test: General Manual , 1958 .

[13]  A Parducci,et al.  The category effect in social judgment: experimental ratings of happiness. , 1988, Journal of personality and social psychology.

[14]  John C. Leggett,et al.  Caste, Class, and Deference in the Research Interview , 1960, American Journal of Sociology.

[15]  Dorothy Watson,et al.  Correcting for Acquiescent Response Bias in the Absence of a Balanced Scale , 1992 .

[16]  C. A. Moser,et al.  Survey Methods in Social Investigation. , 1958 .

[17]  Patrick M. O'Malley,et al.  Yea-Saying, Nay-Saying, and Going to Extremes: Black-White Differences in Response Styles , 1984 .

[18]  Donald R. Lehmann,et al.  Are Three-Point Scales Always Good Enough? , 1972 .

[19]  Catherine E. Ross,et al.  Eliminating defense and agreement bias from measures of the sense of control : A 2×2 index , 1991 .

[20]  A. Dudycha,et al.  Effects of Item Format on Item Discrimination and Difficulty. , 1973 .

[21]  Jon A. Krosnick,et al.  The Measurement of Values in Surveys: A Comparison of Ratings and Rankings , 1985 .

[22]  Jon A. Krosnick,et al.  The Impact of the Gulf War on the Ingredients of Presidential Evaluations: Multidimensional Effects of Political Involvement , 1993, American Political Science Review.

[23]  L. Thurstone.  Attitudes Can Be Measured , 1928, American Journal of Sociology.

[24]  J. Jacoby,et al.  Is There an Optimal Number of Alternatives for Likert Scale Items? Study I: Reliability and Validity , 1971 .

[25]  S. Presser,et al.  Survey Questions: Handcrafting the Standardized Questionnaire. , 1988 .

[26]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[27]  Philip B. Gough,et al.  Grammatical transformations and speed of understanding , 1965 .

[28]  J. Krosnick,et al.  Attitude strength: One construct or many related constructs? , 1993 .

[29]  Dennis J. Aigner,et al.  On the Explanatory Power of Dummy Variable Regressions , 1975 .

[30]  John P. Robinson,et al.  Questions and answers in attitude surveys , 1982 .

[31]  Rivka R. Eifermann,et al.  Negation: A linguistic variable , 1961 .

[32]  R. Lissitz,et al.  Effect of the number of scale points on reliability: A Monte Carlo approach. , 1975 .

[33]  W. Martin.  The Effects of Scaling on the Correlation Coefficient: A Test of Validity , 1973 .

[34]  D. Holt,et al.  The Effects of Offering a Middle Response Option with Opinion Questions , 1980 .

[35]  R. Likert.  “Technique for the Measurement of Attitudes, A” , 2022, The SAGE Encyclopedia of Research Design.

[36]  R. Mason,et al.  Characteristics of Nonopinion and No Opinion Response Groups , 1978 .

[37]  Jon A. Krosnick,et al.  EDUCATION MODERATES SOME RESPONSE EFFECTS IN ATTITUDE MEASUREMENT , 1996 .

[38]  J. N. Bassili.  Meta-judgmental versus operative indexes of psychological attributes: The case of measures of attitude strength. , 1996 .

[39]  P. C. Wason,et al.  The Processing of Positive and Negative Information , 1959 .

[40]  Bruce Buchanan,et al.  WANTED: THE EDUCATION PRESIDENT ISSUE TRESPASSING BY POLITICAL CANDIDATES , 1992 .

[41]  Julian L. Biggers.  Use of a Neutral Point with the Dogmatism Scale , 1977 .

[42]  W. Bielby.  Arbitrary Metrics in Multiple-Indicator Models of Latent Variables , 1986 .

[43]  Paul E. Green,et al.  Rating Scales and Information Recovery—How Many Scales and Response Categories to Use? , 1970 .

[44]  J. Krosnick.  Response strategies for coping with the cognitive demands of attitude measures in surveys , 1991 .

[45]  Jerry C. Olson,et al.  When Do the Measures of Knowledge Measure What We Think They Are Measuring , 1990 .

[46]  Douglas H. Wedell,et al.  Reducing the dependence of clinical judgment on the immediate context: effects of number of categories and type of anchors. , 1990 .

[47]  Jon A. Krosnick,et al.  Psychological determinants of political opinionation , 1990 .

[48]  C. Osgood,et al.  The Measurement of Meaning , 1958 .

[49]  R. H. Finn.  Effects of Some Variations in Rating Scale Characteristics on the Means and Reliabilities of Ratings , 1972 .

[50]  David E. Kanouse,et al.  Controlling for Acquiescence Response Set in Scale Development , 1982 .

[51]  T. Moum,et al.  Yea-saying and mood-of-the-day effects in self-reported quality of life , 1988 .

[52]  Jon A. Krosnick,et al.  Satisficing in surveys: Initial evidence , 1996 .

[53]  D. Campbell,et al.  Response set in the F scale. , 1957, Journal of abnormal psychology.

[54]  E. R. Wembridge.  Obscurities in voting upon measures due to double-negative. , 1918 .