Bias and Changes in Perceived Intensity of Verbal Qualifiers Effected by Scale Orientation

The objective of this study is to examine whether manipulating the orientation of a rating scale impacts on the perceived intensity of the verbal qualifiers. An experiment is designed to assess the perception of intensities of verbal qualifiers in an agreement scale. A first finding is that participants seem to adopt one of two response strategies. Those who show the ’extreme null-point strategy’ report perceived intensities that monotonically increase along with the scale from ’fully disagree’ to ’fully agree’. However, other respondents seem to adopt a ’middle null-point strategy’, where the highest perceived intensity coincides with both extreme qualifiers of the scale and the lowest intensity is experienced for qualifiers near the middle. An orientation effect is supported for the ’extreme null-point strategy’ group, and manifests itself in less agreement about the intensity of the qualifiers when placed on a decremental scale (e.g. fully agree - rather agree - neutral - rather disagree - fully disagree) as opposed to an incremental scale (e.g. fully disagree - rather disagree - neutral - rather agree - fully agree). Next, the existence of a primacy-effect, an orientation effect found in previous research, was tested by means of a web survey-experiment and is rejected in favour of a more differentiated effect.

[1]  J. Kidd Management Teams: Why They Succeed or Fail , 1982 .

[2]  Jiyoung Kwahk,et al.  A systematic method for analyzing magnitude estimation data , 1999 .

[3]  F. Strack,et al.  Thinking, Judging, and Communicating: A Process Account of Context Effects in Attitude Surveys , 1987 .

[4]  G. F. Bishop,et al.  Response-Order Effects and the Early Gallup Split-Ballots , 2001 .

[5]  Michael H. Birnbaum,et al.  How to show that 9 > 221 : Collect judgments in a between-subjects design , 1999 .

[6]  Milton Lodge,et al.  Magnitude Scaling: Quantitative Measurement of Opinions , 1981 .

[7]  G A Gescheider,et al.  Psychophysical scaling. , 1988, Annual review of psychology.

[8]  Charles S. Reichardt,et al.  Justifying the use and increasing the power of a t test for a randomized experiment with a convenience sample. , 1999 .

[9]  Michael H. Birnbaum,et al.  Loci of contextual effects in judgment. , 1982 .

[10]  W. Belson,et al.  The effects of reversing the presentation order of verbal rating scales , 1966 .

[11]  J. Falmagne Elements of psychophysical theory , 1985 .

[12]  S. S. Stevens Mathematics, measurement, and psychophysics. , 1951 .

[13]  K. Myers,et al.  Ten-year review of rating scales. I: overview of scale functioning, psychometric properties, and selection. , 2002, Journal of the American Academy of Child and Adolescent Psychiatry.

[14]  S. S. Stevens Sensory scales of taste intensity , 1969 .

[15]  S. Link,et al.  Bias in Quantifying Judgments , 1989 .

[16]  Ron Garland A Comparison of Three Forms of the Semantic Differential , 1990 .

[17]  Chung-Ping Cheng,et al.  Effects of Response Order on Likert-Type Scales , 2000 .

[18]  Sean Hammond,et al.  Research Methods in Psychology , 1994 .

[19]  J. Krosnick,et al.  AN EVALUATION OF A COGNITIVE THEORY OF RESPONSE-ORDER EFFECTS IN SURVEY MEASUREMENT , 1987 .

[20]  Jason C. Chan Response-Order Effects in Likert-Type Scales , 1991 .

[21]  Joeri Hofmans,et al.  Context in category scales: is “fully agree” equal to twice agree? , 2006 .