An Enquiry into the Method of Paired Comparison: Reliability, Scaling, and Thurstone's Law of Comparative Judgment

The method of paired comparisons is used to measure individuals' preference orderings of items presented to them as discrete binary choices. This paper reviews the theory and application of the paired comparison method, describes a new computer program available for eliciting the choices, and presents an analysis of methods for scaling paired choice data to estimate an interval scale measure of preference. A new procedure for isolating an individual's inconsistent choices is described. Using data from five empirical studies, the reliability of respondents' paired choices is assessed using measures of internal reliability, choice consistency, and test-retest reliability.

[1]  William R. Dillon,et al.  Capturing Individual Differences in Paired Comparisons: An Extended BTL Model Incorporating Descriptor Variables , 1993 .

[2]  H. A. David,et al.  The method of paired comparisons , 1966 .

[3]  G. Fechner.  Elemente der Psychophysik , 1998 .

[4]  Moshe Ben-Akiva,et al.  Discrete Choice Analysis: Theory and Application to Travel Demand , 1985 .

[5]  E. Titchener,et al.  Experimental Psychology: A Manual of Laboratory Practice. , 1901 .

[6]  R. Duncan Luce,et al.  Individual Choice Behavior , 1959 .

[7]  T. Brown,et al.  Economic Valuation by the Method of Paired Comparison, with Emphasis on Evaluation of the Transitivity Axiom , 1998 .

[8]  Thomas C. Brown,et al.  Multiple Good Valuation , 2003 .

[9]  M. Kendall,et al.  ON THE METHOD OF PAIRED COMPARISONS , 1940 .

[10]  Roger R. Davidson,et al.  A Bibliography on the Method of Paired Comparisons , 1973 .

[11]  Thomas C. Brown,et al.  The judged seriousness of an environmental loss is a matter of what caused it , 2005 .

[12]  Thomas C. Brown,et al.  Judged seriousness of environmental losses: reliability and cause of loss , 2002 .

[13]  Charles R. Plott,et al.  Rational Individual Behavior in Markets and Social Choice Processes: the Discovered Preference Hypothesis , 1993 .

[14]  John B. Loomis,et al.  Applying a Method of Paired Comparisons to Measure Economic Values for Multiple Goods Sets , 2002, Journal of Agricultural and Applied Economics.

[15]  C. Manski.  The structure of random utility models , 1977 .

[16]  L. V. Jones,et al.  The measurement and prediction of judgment and choice. , 1970 .

[17]  L. Thurstone.  A law of comparative judgment. , 1994 .

[18]  Joseph L. Zinnes,et al.  Theory and Methods of Scaling. , 1958 .

[19]  T. Brown,et al.  Measuring dispositions for lexicographic preferences of environmental goods: integrating economics, psychology and ethics , 2003 .

[20]  D. Hendel.  Intransitivity on Paired-Comparisons Instruments , 1977 .

[21]  John M. Rose,et al.  Applied Choice Analysis: A Primer , 2005 .

[22]  Robin Gregory,et al.  Valuing environmental resources: A constructive approach , 1993 .

[23]  T. Ll.  Psychophysical analysis. By L. L. Thurstone, 1927. , 1987 .

[24]  Edward Bradford Titchener,et al.  Experimental psychology: A manual of laboratory practice, Vol 1: Qualitative experiments: Part II. Instructor's manual. , 1901 .

[25]  Gregory J. Buhyoff,et al.  Estimating psychological disutility from damaged forest stands. , 1978 .

[26]  J. Busemeyer,et al.  Extending the Bounds of Rationality: Evidence and Theories of Preferential Choice , 2006 .

[27]  L. Thurstone.  The method of paired comparisons for social values , 1927 .

[28]  P. Bell,et al.  The Influence of Attitude Priming and Social Responsibility on the Valuation of Environmental Public Goods Using Paired Comparisons , 1999 .

[29]  D. McFadden.  Conditional logit analysis of qualitative choice behavior , 1972 .