Analysis of pairwise comparison matrices: an empirical research

Pairwise comparison (PC) matrices are used in multi-attribute decision problems (MADM) in order to express the preferences of the decision maker. Our research focused on testing various characteristics of PC matrices. In a controlled experiment with university students (N=227) we have obtained 454 PC matrices. The cases have been divided into 18 subgroups according to the key factors to be analyzed. Our team conducted experiments with matrices of different size given from different types of MADM problems. Additionally, the matrix elements have been obtained by different questioning procedures differing in the order of the questions. Results are organized to answer five research questions. Three of them are directly connected to the inconsistency of a PC matrix. Various types of inconsistency indices have been applied. We have found that the type of the problem and the size of the matrix had impact on the inconsistency of the PC matrix. However, we have not found any impact of the questioning order. Incomplete PC matrices played an important role in our research. The decision makers behavioral consistency was as well analyzed in case of incomplete matrices using indicators measuring the deviation from the final order of alternatives and from the final score vector.

[1]  Lajos Rónyai,et al.  On optimal completion of incomplete pairwise comparison matrices , 2010, Math. Comput. Model..

[2]  Catherine K. Murphy,et al.  LIMITS ON THE ANALYTIC HIERARCHY PROCESS FROM ITS CONSISTENCY INDEX , 1993 .

[3]  P. Moran.  On the method of paired comparisons. , 1947, Biometrika.

[4]  Saul I. Gass,et al.  Characteristics of positive reciprocal matrices in the analytic hierarchy process , 2002, J. Oper. Res. Soc..

[5]  Saul I. Gass,et al.  Singular value decomposition in AHP , 2004, Eur. J. Oper. Res..

[6]  P. Harker.  Incomplete pairwise comparisons in the analytic hierarchy process , 1987 .

[7]  Waldemar W. Koczkodaj,et al.  An LP-based inconsistency monitoring of pairwise comparison matrices , 2011, Math. Comput. Model..

[8]  Carlos A. Bana e Costa,et al.  A critical analysis of the eigenvalue method used to derive priorities in AHP , 2008, Eur. J. Oper. Res..

[9]  Waldemar W. Koczkodaj,et al.  Using consistency-driven pairwise comparisons in knowledge-based systems , 1997, CIKM '97.

[10]  Matteo Brunelli,et al.  CHARACTERIZING PROPERTIES FOR INCONSISTENCY INDICES IN THE AHP , 2011 .

[11]  Tamás Rapcsák,et al.  On Saaty’s and Koczkodaj’s inconsistencies of pairwise comparison matrices , 2008, J. Glob. Optim..

[12]  Matthias Ehrgott,et al.  Multiple criteria decision analysis: state of the art surveys , 2005 .

[13]  Saul I. Gass,et al.  Tournaments, transitivity and pairwise comparison matrices , 1998, J. Oper. Res. Soc..

[14]  Gerzson Kéri.  On qualitatively consistent, transitive and contradictory judgment matrices emerging from multiattribute decision procedures , 2011, Central Eur. J. Oper. Res..

[15]  János Fülöp,et al.  On pairwise comparison matrices that can be made consistent by the modification of a few elements , 2011, Central Eur. J. Oper. Res..

[16]  M. T. Lamata,et al.  A new measure of consistency for positive reciprocal matrices , 2003 .

[17]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[18]  T. Saaty,et al.  Why the magic number seven plus or minus two , 2003 .

[19]  József Temesi,et al.  Consistency of the decision-maker in pair-wise comparisons , 2006 .

[20]  József Temesi,et al.  Pairwise comparison matrices and the error-free property of the decision maker , 2011, Central Eur. J. Oper. Res..

[21]  W. W. Koczkodaj.  A new definition of consistency of pairwise comparisons , 1993 .

[22]  R. Ross.  Optimum orders for the presentation of pairs in the method of paired comparisons. , 1934 .

[23]  M. Kendall,et al.  ON THE METHOD OF PAIRED COMPARISONS , 1940 .

[24]  X. Zeshui,et al.  A consistency improving method in the analytic hierarchy process , 1999, Eur. J. Oper. Res..

[25]  José María Moreno-Jiménez,et al.  The geometric consistency index: Approximated thresholds , 2003, Eur. J. Oper. Res..

[26]  Zeshui Xu,et al.  A consistency improving method in the analytic hierarchy process , 1999, Eur. J. Oper. Res..

[27]  Pedro Linares Llamas.  Are inconsistent decisions better? An experiment with pairwise comparisons , 2009 .

[28]  Eng Ung Choo,et al.  A common framework for deriving preference values from pairwise comparison matrices , 2004, Comput. Oper. Res..

[29]  T. Saaty,et al.  The Analytic Hierarchy Process , 1985 .