Do Category Rating Scales Produce Biased Preference Weights for a Health Index?

Category rating scales are frequently used to measure social preferences in health decision models. The use of category rating scale methods has been questioned because subjects allegedly attempt to use rating scale categories equally often. This results in scale values that are contingent upon the context in which a stimulus is presented. In Experiment 1, subjects were randomly divided into five groups that rated a series of health case descriptions using the category method. Extensive pretesting had established the scale values for the stimuli. For one group, all values of the stimuli were evenly distributed across the response scale (as determined by pretests). For a second group, only items from the top of the scale were used; for a third group, only items with low values were used; for the fourth group items with low and high values (but not middle range) were presented; and for the fifth, items with medium values were presented. The results demonstrated that the values for items remain essentially the same, regardless of the group in which they are presented. These results were confirmed and extended in a second experiment. Experiment 3 demonstrated that subjects will attempt to use categories equally often when the stimuli are not health case descriptions but only when the response continuum and endpoints of the rating scale are poorly defined. It is suggested that rating scales can provide valid and reliable results if the response continuum is made clear to subjects.

[1]  S. S. Stevens Issues in psychophysical measurement. , 1971 .

[2]  R. Sekuler,et al.  The invalidity of “invalid results from the method of constant stimuli”: A common artifact in the methods of psychophysics , 1971 .

[3]  R. Kirk Experimental Design: Procedures for the Behavioral Sciences , 1970 .

[4]  S. S. Stevens,et al.  Ratio scales and category scales for a dozen perceptual continua. , 1957, Journal of experimental psychology.

[5]  D L Patrick,et al.  Toward an operational definition of health. , 1973, Journal of health and social behavior.

[6]  Allen Parducci,et al.  Range-frequency compromise in judgment. , 1963 .

[7]  Robert L. Berg,et al.  Health Status Indexes. , 1975 .

[8]  S S Stevens,et al.  To Honor Fechner and Repeal His Law: A power function, not a log function, describes the operating characteristic of a sensory system. , 1961, Science.

[9]  A. Parducci The relativism of absolute judgements. , 1968, Scientific American.

[10]  Norman H. Anderson,et al.  How functional measurement can yield validated interval scales of mental quantities. , 1976 .

[11]  Gustav Theodor Fechner,et al.  Elements of psychophysics , 1966 .

[12]  George W. Torrance,et al.  Social preferences for health states: An empirical evaluation of three measurement techniques , 1976 .

[13]  M. M. Chen,et al.  Analysis of a tuberculin testing program using a health status index , 1972 .

[14]  R M Kaplan,et al.  Health Status Index: Category Rating versus Magnitude Estimation for Measuring Levels of Well-Being , 1979, Medical care.

[15]  J. W. Bush,et al.  A Health-Status Index and its Application to Health-Services Outcomes , 1970, Oper. Res..

[16]  L L THURSTONE,et al.  The measurement of values. , 1960, Psychological review.

[17]  M. Weinstein,et al.  Foundations of cost-effectiveness analysis for health and medical practices. , 1977, The New England journal of medicine.

[18]  J. M. Grossberg,et al.  Clinical psychophysics: applications of ratio scaling and signal detection methods to research on pain, fear, drugs, and medical decision making. , 1978, Psychological bulletin.

[19]  Jaap Van Brakel,et al.  Foundations of measurement , 1983 .

[20]  D L Patrick,et al.  Methods for measuring levels of well-being for a health status index. , 1973, Health services research.

[21]  S. D. Roberts Cost-effective oxygen therapy. , 1980, Annals of internal medicine.

[22]  D. Weiss Averaging: An empirical validity criterion for magnitude estimation , 1972 .

[23]  M. P. Friedman,et al.  HANDBOOK OF PERCEPTION , 1977 .

[24]  H. Sintonen An approach to measuring and valuing health states. , 1981, Social science & medicine. Medical economics.

[25]  David J. Weiss,et al.  Quantifying private events: A functional measurement analysis of equisection , 1975 .

[26]  N. Anderson Chapter 8 – ALGEBRAIC MODELS IN PERCEPTION* , 1974 .

[27]  L M Ward,et al.  Category judgments of loudnesses in the absence of an experimenter-induced identification function: sequential effects and power-function fit. , 1972, Journal of experimental psychology.

[28]  S. S. Stevens On the psychophysical law. , 1957, Psychological review.