Modeling within-subject dependencies in ordinal paired comparison data

This paper presents two probabilistic models based on the logistic and the normal distribution for the analysis of dependencies in individual paired comparison judgments. It is argued that a core assumption of latent class choice models, independence of individual decisions, may not be well-suited for the analysis of paired comparison data. Instead, the analysis and interpretation of paired comparison data may be much simplified by allowing for within-person dependencies that result from repeated evaluations of the same options in different pairs. Moreover, by relating dependencies among the individual-level responses to (in)consistencies in the judgmental process, we show that the proposed graded paired comparison models reduce to ranking models under certain conditions. Three applications are presented to illustrate the approach.

[1]  L Sjöberg,et al.  Successive intervals scaling of paired comparisons , 1967, Psychometrika.

[2]  P. Acker.  Transitivity revisited , 1990 .

[3]  Alan Agresti,et al.  Analysis of Ordinal Paired Comparison Data , 1992 .

[4]  Jeroen K. Vermunt,et al.  lEM : Log-linear and event history analysis with missing data using the EM algorithm , 1993 .

[5]  M. Kendall,et al.  Part 1: Factor Analysis as a Statistical Technique , 1950 .

[6]  M. Kendall,et al.  Rank Correlation Methods , 1949 .

[7]  P. Moran.  Recent Developments in Ranking Theory , 1950 .

[8]  R. Duncan Luce,et al.  Individual Choice Behavior , 1959 .

[9]  Harry Joe,et al.  Majorization, entropy and paired comparisons , 1988 .

[10]  A. Formann.  Linear Logistic Latent Class Analysis for Polytomous Data , 1992 .

[11]  L. V. Jones,et al.  The measurement and prediction of judgment and choice. , 1970 .

[12]  G Tutz,et al.  Bradley-Terry-Luce models with an ordered response , 1986 .

[13]  L. Thurstone.  A law of comparative judgment. , 1994 .

[14]  R. Davidson.  On Extending the Bradley-Terry Model to Accommodate Ties in Paired Comparison Experiments , 1970 .

[15]  H. A. David,et al.  The method of paired comparisons , 1966 .

[16]  S. Fienberg,et al.  Log linear representation for paired and multiple comparisons models , 1976 .

[17]  H. A. David,et al.  Ties in Paired-Comparison Experiments Using a Modified Thurstone-Mosteller Model , 1960 .

[18]  M. Kendall.  Ranks and measures , 1962 .

[19]  S. Yakowitz,et al.  On the Identifiability of Finite Mixtures , 1968 .

[20]  W. Cook,et al.  Ordinal Ranking with Intensity of Preference , 1985 .

[21]  U. Böckenholt.  Thurstonian representation for partial ranking data , 1992 .

[22]  H. Morrison.  Testable conditions for triads of paired comparison choices , 1963 .

[23]  William R. Dillon,et al.  Capturing Individual Differences in Paired Comparisons: An Extended BTL Model Incorporating Descriptor Variables , 1993 .

[24]  L. A. Goodman.  Simple Models for the Analysis of Association in Cross-Classifications Having Ordered Categories , 1979 .

[25]  C. Park.  A seven-point scale and a decision-maker's simplifying choice strategy: An operationalized satisficing-plus model , 1978 .

[26]  L. L. Thurstone,et al.  An internal consistency check for scale values determined by the method of successive intervals , 1952 .

[27]  Ralph A. Bradley,et al.  RANKING IN TRIPLE COMPARISONS , 1959 .

[28]  R. A. Bradley,et al.  RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS THE METHOD OF PAIRED COMPARISONS , 1952 .

[29]  A. Formann.  Constrained latent class models: Some further applications† , 1989 .

[30]  A. Tversky.  Intransitivity of preferences. , 1969 .

[31]  Yoshio Takane,et al.  Analysis of Covariance Structures and Probabilistic Binary Choice Data , 1989 .

[32]  Shelby J. Haberman,et al.  A Stabilized Newton-Raphson Algorithm for Log-Linear Models for Frequency Tables Derived by Indirect Observation , 1988 .

[33]  P. Holland,et al.  Discrete Multivariate Analysis. , 1976 .

[34]  Wolfgang Gaul,et al.  Analysis of choice behaviour via probabilistic ideal point and vector models , 1986 .

[35]  Jan de Leeuw,et al.  On the relationship between item response theory and factor analysis of discretized variables , 1987 .

[36]  Stephen E. Fienberg,et al.  Discrete Multivariate Analysis: Theory and Practice , 1976 .

[37]  J. Douglas Carroll,et al.  Toward a new paradigm for the study of multiattribute choice behavior: Spatial and discrete modeling of pairwise preferences. , 1991 .

[38]  H. Halff.  Choice theories for differentially comparable alternatives , 1976 .

[39]  S. S. Stevens,et al.  Psychophysics: Introduction to Its Perceptual, Neural and Social Prospects , 1975 .

[40]  B. Mellers,et al.  Similarity and Choice. , 1994 .

[41]  Yoshio Takane,et al.  MAXIMUM LIKELIHOOD ESTIMATION IN THE GENERALIZED CASE OF THURSTONE'S MODEL OF COMPARATIVE JUDGMENT , 1980 .

[42]  Suzanne Winsberg,et al.  A thurstonian pairwise choice model with univariate and multivariate spline transformations , 1993 .

[43]  R. A. Bradley,et al.  Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons , 1952 .

[44]  Donald L. Rumelhart,et al.  Similarity between stimuli: An experimental test of the Luce and Restle choice models. , 1971 .