Comparative judgments as an alternative to ratings: identifying the scale origin.

Although comparative judgment methods have a number of distinct advantages over ratings, they share one common problem: On the basis of comparative judgments, it is not possible to recover the origin of item evaluations. One item may be judged more positively than another, but this result does not allow any conclusions about whether either of the items are attractive or unattractive. This article discusses the implications of this limitation for the interpretation of individual differences in comparative judgments. It also presents 3 different methods that may allow determination of the scale origin using a nested model comparison approach. An application illustrates the proposed approach as well as the benefits of determining the scale origin in understanding value judgments.

[1]  David V. Budescu,et al.  A review of human linguistic probability processing: General principles and empirical evidence , 1995, The Knowledge Engineering Review.

[2]  P. Bentler,et al.  Significance Tests and Goodness of Fit in the Analysis of Covariance Structures , 1980 .

[3]  M. Kaplan,et al.  Human judgment and decision processes , 1975 .

[4]  N. Anderson.  Foundations of information integration theory , 1981 .

[5]  Henry E. Brady.  Factor and ideal point analysis for interpersonally incomparable data , 1989 .

[6]  U Böckenholt.  Hierarchical modeling of paired comparison data. , 2001, Psychological methods.

[7]  L. V. Jones,et al.  The Rational Origin for Measuring Subjective Values , 1957 .

[8]  R. Luce.  Utility of Gains and Losses: Measurement-Theoretical and Experimental Approaches , 2000 .

[9]  R. Duncan Luce,et al.  Individual Choice Behavior , 1959 .

[10]  A. Tversky,et al.  Prospect Theory : An Analysis of Decision under Risk Author ( s ) : , 2007 .

[11]  Alan Nash,et al.  Foundations of information integration , 2006 .

[12]  Albert Maydeu-Olivares,et al.  Limited information estimation and testing of Thurstonian models for preference data , 2002, Math. Soc. Sci..

[13]  Gregory Camilli,et al.  Origin of the scaling constant d = 1.7 in item response theory. , 1994 .

[14]  D. Hedeker,et al.  MIXOR: a computer program for mixed-effects ordinal regression analysis. , 1996, Computer methods and programs in biomedicine.

[15]  Christopher K. Hsee,et al.  Preference Reversals between Joint and Separate Evaluations of Options: A Review and Theoretical Analysis , 1999 .

[16]  U. Böckenholt,et al.  Hierarchical modeling of paired comparison data. , 2001, Psychological methods.

[17]  William L. Rankin,et al.  A comparison of ranking and rating procedures for value system measurement , 1980 .

[18]  Ulf Böckenholt,et al.  Two-level linear paired comparison models: estimation and identifiability issues , 2002, Math. Soc. Sci..

[19]  S. Schwartz,et al.  Toward a theory of the universal content and structure of values: Extensions and cross-cultural replications. , 1990 .

[20]  P. Moran.  On the method of paired comparisons. , 1947, Biometrika.

[21]  H. Gulliksen.  Measurement of subjective values , 1956 .

[22]  D. Ariely,et al.  Constructing Stable Preferences: A Look Into Dimensions of Experience and Their Impact on Preference Stability , 1999 .

[23]  L. V. Jones,et al.  A rational origin obtained by the method of contingent paired comparisons , 1960 .

[24]  R. J. Mokken,et al.  Handbook of modern item response theory , 1997 .

[25]  W. Chan,et al.  ANALYZING IPSATIVE DATA IN PSYCHOLOGICAL RESEARCH , 2003 .

[26]  James Shanteau,et al.  5 – AN INFORMATION-INTEGRATION ANALYSIS OF RISKY DECISION MAKING , 1975 .

[27]  A. Tversky,et al.  Conjoint-measurement analysis of composition rules in psychology. , 1971 .

[28]  L. Thurstone.  A law of comparative judgment. , 1994 .

[29]  Jacqueline J. Meulman,et al.  New Developments in Psychometrics. , 2003 .

[30]  Ian H. Langford,et al.  A User’s Guide to MLwiN, Version 2.10 , 2000 .

[31]  Raymond B. Cattell,et al.  Psychological measurement: normative, ipsative, interactive. , 1944 .

[32]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[33]  J. Berge.  A Legitimate Case of Component Analysis of Ipsative Measures, and Partialling the Mean as an Alternative to Ipsatization , 1999 .

[34]  Albert Maydeu-Olivares.  On Thurstone’s Model for Paired Comparisons and Ranking Data , 2003 .

[35]  D. Kahneman.  Maps of Bounded Rationality: Psychology for Behavioral Economics , 2003 .

[36]  Willem J. Heiser,et al.  Multidimensional mapping of preference data , 1981 .

[37]  H. A. David,et al.  The method of paired comparisons , 1966 .

[38]  Harold Gulliksen,et al.  Measurement of subjective values , 1956 .

[39]  S. Schwartz,et al.  Toward A Universal Psychological Structure of Human Values , 1987 .

[40]  R. Tsai,et al.  Remarks on the identifiability of thurstonian paired comparison models under multiple judgment , 2003 .

[41]  A. Tversky,et al.  Choices, Values, and Frames , 2000 .

[42]  Jon A. Krosnick,et al.  The Measurement of Values in Surveys: A Comparison of Ratings and Rankings , 1985 .

[43]  A. Tversky.  Intransitivity of preferences. , 1969 .