Rank reversals in multi-criteria decision analysis with statistical modelling of ratio-scale pairwise comparisons

In multi-criteria decision analysis, the overall performance of decision alternatives is evaluated with respect to several, generally conflicting decision criteria. One approach to perform the multi-criteria decision analysis is to use ratio-scale pairwise comparisons concerning the performance of decision alternatives and the importance of decision criteria. In this approach, a classical problem has been the phenomenon of rank reversals. In particular, when a new decision alternative is added to a decision problem, and while the assessments concerning the original decision alternatives remain unchanged, the new alternative may cause rank reversals between the utility estimates of the original decision alternatives. This paper studies the connections between rank reversals and the potential inconsistency of the utility assessments in the case of ratio-scale pairwise comparisons data. The analysis was carried out by recently developed statistical modelling techniques so that the inconsistency of the assessments was measured according to statistical estimation theory. Several type of decision problems were analysed and the results showed that rank reversals caused by inconsistency are natural and acceptable. On the other hand, rank reversals caused by the traditional arithmetic-mean aggregation rule are not in line with the ratio-scale measurement of utilities, whereas geometric-mean aggregation does not cause undesired rank reversals.

[1]  Jyrki Kangas,et al.  Uncertainty in Expert Predictions of the Ecological Consequences of Forest Plans , 1996 .

[2]  F. Lootsma.  Multi-Criteria Decision Analysis via Ratio and Difference Judgement , 1999 .

[3]  T. Saaty,et al.  The Analytic Hierarchy Process , 1985 .

[4]  P. D. Jong.  A statistical approach to Saaty's scaling method for priorities , 1984 .

[5]  A. Rapoport,et al.  A Comparison of the Eigenvalue Method and The Geometric Mean Procedure for Ratio Scaling , 1986 .

[6]  T. L. Saaty.  A Scaling Method for Priorities in Hierarchical Structures , 1977 .

[7]  Thomas L. Saaty,et al.  On the relativity of relative measures - accommodating both rank preservation and rank reversals in the AHP , 2000, Eur. J. Oper. Res..

[8]  P. Leskinen.  Measurement scales and scale independence in the Analytic Hierarchy Process , 2000 .

[9]  O. I. Larichev,et al.  Comments on Barzilai and Lootsma , 1997 .

[10]  Evangelos Triantaphyllou,et al.  Two new cases of rank reversals when the AHP and some of its additive variants are used that do not occur with the multiplicative AHP , 2001 .

[11]  W. Wedley,et al.  Ambiguous Criteria Weights in AHP: Consequences and Solutions* , 1989 .

[12]  D. Schmoldt,et al.  The Analytic Hierarchy Process in Natural Resource and Environmental Decision Making , 2001, Managing Forest Ecosystems.

[13]  F. B. Vernadat,et al.  Decisions with Multiple Objectives: Preferences and Value Tradeoffs , 1994 .

[14]  Thomas L. Saaty,et al.  How to Make a Decision: The Analytic Hierarchy Process , 1990 .

[15]  Jonathan Barzilai,et al.  POWER RELATIONS AND GROUP AGGREGATION IN THE MULTIPLICATIVE AHP AND SMART , 1997 .

[16]  William C. Wedley,et al.  Clustering, dependence and ratio scales in AHP: Rank reversals and incorrect priorities with a single criterion , 1993 .

[17]  C. Genest,et al.  A statistical look at Saaty's method of estimating pairwise preferences expressed on a ratio scale , 1994 .

[18]  Jyrki Kangas,et al.  Analyzing uncertainties in experts' opinions of forest plan performance , 1997 .

[19]  P. Leskinen,et al.  Regression Methods for Pairwise Comparison Data , 2001 .

[20]  Thomas L. Saaty,et al.  Multicriteria Decision Making: The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation , 1990 .

[21]  J. Barzilai.  Deriving weights from pairwise comparison matrices , 1997 .

[22]  Luis G. Vargas,et al.  Reply to “remarks on the analytic hierarchy process” by J. S. Dyer , 1990 .

[23]  Antonie Stam,et al.  On multiplicative priority rating methods for the AHP , 2003, Eur. J. Oper. Res..

[24]  Valerie Belton,et al.  On a short-coming of Saaty's method of analytic hierarchies , 1983 .

[25]  J. Barzilai,et al.  Ahp Rank Reversal, Normalization and Aggregation Rules , 1994 .

[26]  Stan Schenkerman.  Avoiding rank reversal in AHP decision-support models , 1994 .

[27]  Jyrki Kangas,et al.  Analysing uncertainties of interval judgment data in multiple-criteria evaluation of forest plans , 1998 .

[28]  R. L. Keeney,et al.  Decisions with Multiple Objectives: Preferences and Value Trade-Offs , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[29]  H. Raiffa,et al.  Decisions with Multiple Objectives , 1993 .

[30]  G. Crawford,et al.  A note on the analysis of subjective judgment matrices , 1985 .