Does AHP help us make a choice? An experimental evaluation

In this paper, we use experimental economics methods to test how well Analytic Hierarchy Process (AHP) fares as a choice support system in a real decision problem. AHP provides a ranking that we statistically compare with three additional rankings given by the subjects in the experiment: one at the beginning, one after providing AHP with the necessary pair-wise comparisons and one after learning the ranking provided by AHP. While the rankings vary widely across subjects, we observe that for each individual all four rankings are similar. Hence, subjects are consistent and AHP is, for the most part, able to replicate their rankings. Furthermore, while the rankings are similar, we do find that the AHP ranking helps the decision makers reformulate their choices by taking into account suggestions made by AHP.

[1]  Metin Dagdeviren,et al.  Using the analytic network process (ANP) in a SWOT analysis - A case study for a textile firm , 2007, Inf. Sci..

[2]  J. Barzilai.  Notes on the Analytic Hierarchy Process , 2000 .

[3]  J. Dyer.  Remarks on the analytic hierarchy process , 1990 .

[4]  Cathal M. Brugha,et al.  Phased multicriteria preference finding , 2004, Eur. J. Oper. Res..

[5]  D. Y. Sha,et al.  Supply chain network design: partner selection and production/distribution planning using a systematic model , 2006, J. Oper. Res. Soc..

[6]  Karen Donohue,et al.  Experimental Economics and Supply-Chain Management , 2002 .

[7]  Joachim Weimann,et al.  Experiments in Environmental Economics and Some Close Relatives , 2006 .

[8]  Eric J. Johnson,et al.  Adaptive Strategy Selection in Decision Making. , 1988 .

[9]  Luis G. Vargas.  An overview of the analytic hierarchy process and its applications , 1990 .

[10]  Saul I. Gass,et al.  The Analytic Hierarchy Process - An Exposition , 2001, Oper. Res..

[11]  H. A. Donegan,et al.  Comparison of Prioritization Techniques Using Interhierarchy Mappings , 1995 .

[12]  Pekka Korhonen,et al.  Performance of the AHP in comparison of gains and losses , 2003 .

[13]  Karen Donohue,et al.  Special Issue: Experimental Economics in Practice: Experimental Economics and Supply-Chain Management , 2002, Interfaces.

[14]  Adel Guitouni,et al.  Une typologie des méthodes multicritères: Proposition d'un cadre méthodologique , 2007, INFOR Inf. Syst. Oper. Res..

[15]  John M. Hassell,et al.  The sensitivity of the analytic hierarchy process to alternative scale and cue presentations , 1997 .

[16]  Dean S. Karlan,et al.  Using Experimental Economics to Measure Social Capital and Predict Financial Decisions , 2005 .

[17]  R. Hämäläinen,et al.  On the measurement of preferences in the analytic hierarchy process , 1997 .

[18]  Thomas L. Saaty,et al.  Rank from comparisons and from ratings in the analytic hierarchy/network processes , 2006, Eur. J. Oper. Res..

[19]  Sushil Kumar,et al.  Analytic hierarchy process: An overview of applications , 2006, Eur. J. Oper. Res..

[20]  Valerie Belton,et al.  On a short-coming of Saaty's method of analytic hierarchies , 1983 .

[21]  Lisa Simpson,et al.  Do Decision Makers Know What They Prefer?: MAVT and ELECTRE II , 1996 .

[22]  L. C. Leung,et al.  Implementing the balanced scorecard using the analytic hierarchy process & the analytic network process , 2006, J. Oper. Res. Soc..

[23]  Mehrdad Tamiz,et al.  An integrated queuing and multi-objective bed allocation model with application to a hospital in China , 2009, J. Oper. Res. Soc..

[24]  Alessio Ishizaka,et al.  Influence of aggregation and measurement scale on ranking a compromise alternative in AHP , 2009, J. Oper. Res. Soc..

[25]  Pedro Linares Llamas.  Are inconsistent decisions better? An experiment with pairwise comparisons , 2009 .

[26]  Eelko Huizingh,et al.  Extending the applicability of the analytic hierarchy process , 1997 .

[27]  Madjid Tavana,et al.  A priority assessment multi-criteria decision model for human spaceflight mission planning at NASA , 2006, J. Oper. Res. Soc..

[28]  N. K. Kwak,et al.  Information resource planning for a health-care system using an AHP-based goal programming method , 1999, J. Oper. Res. Soc..

[29]  Laboratory experimentation in economic research: An introduction to psychologists and marketers , 1996 .

[30]  Carlos A. Bana e Costa,et al.  A critical analysis of the eigenvalue method used to derive priorities in AHP , 2008, Eur. J. Oper. Res..

[31]  William Ho,et al.  Integrated analytic hierarchy process and its applications - A literature review , 2008, Eur. J. Oper. Res..

[32]  G Gigerenzer,et al.  Reasoning the fast and frugal way: models of bounded rationality. , 1996, Psychological review.

[33]  R. Keeney,et al.  Eliciting public values for complex policy decisions , 1990 .

[34]  H. A. Donegan,et al.  A New Approach to Ahp Decision‐Making , 1992 .

[35]  Bhallamudi Ravi,et al.  Web based casting supplier evaluation using analytical hierarchy process , 2001, J. Oper. Res. Soc..

[36]  R. Hämäläinen,et al.  An Experiment on the Numerical Modelling of Verbal Ratio Statements , 1997 .

[37]  Alan Pearman,et al.  Model choice in multicriteria decision aid , 1997 .

[38]  Ido Millet,et al.  The Effectiveness of Alternative Preference Elicitation Methods in the Analytic Hierarchy Process , 1997 .

[39]  Thomas L. Saaty,et al.  The hierarchon : a dictionary of hierarchies , 1996 .

[40]  Charles R. Johnson,et al.  Right-left asymmetry in an eigenvector ranking procedure , 1979 .

[41]  Camille Salinesi,et al.  MCDM Techniques Selection Approaches: State of the Art , 2007, 2007 IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making.

[42]  Anja Dieckmann,et al.  Compensatory versus noncompensatory models for predicting consumer preferences , 2009, Judgment and Decision Making.

[43]  Matthew J. Liberatore,et al.  The analytic hierarchy process in medical and health care decision making: A literature review , 2008, Eur. J. Oper. Res..

[44]  Alessio Ishizaka,et al.  Analytic Hierarchy Process and Expert Choice: Benefits and limitations , 2009, OR Insight.

[45]  B. S. Ahn,et al.  ERP system selection using a simulation-based AHP approach: a case of Korean homeshopping company , 2008, J. Oper. Res. Soc..

[46]  B. Hobbs,et al.  Multicriteria methods for resource planning: an experimental comparison , 1994 .

[47]  S. Wheeler,et al.  An analysis of combined arms teaming for the Australian defence force , 2006, J. Oper. Res. Soc..

[48]  M. Degroot,et al.  Measuring utility by a single-response sequential method. , 1964, Behavioral science.

[49]  Stanislav Karapetrovic,et al.  A quality control approach to consistency paradoxes in AHP , 1999, Eur. J. Oper. Res..

[50]  Miroslaw Kwiesielewicz,et al.  Inconsistent and contradictory judgements in pairwise comparison method in the AHP , 2004, Comput. Oper. Res..

[51]  R. D. Holder,et al.  Response to Holder's Comments on the Analytic Hierarchy Process: Response to the Response , 1991 .

[52]  T. Saaty,et al.  The Analytic Hierarchy Process , 1985 .

[53]  Thomas L. Saaty,et al.  Making and validating complex decisions with the AHP/ANP , 2005 .

[54]  Cathal M. Brugha,et al.  Relative measurement and the power function , 2000, Eur. J. Oper. Res..

[55]  Jung P. Shim,et al.  Bibliographical research on the Analytic Hierarchy Process (AHP) , 1989 .

[56]  Jiju Antony,et al.  Application of stochastic analytic hierarchy process within a domestic appliance manufacturer , 2007, J. Oper. Res. Soc..

[57]  Adel Guitouni,et al.  Tentative guidelines to help choosing an appropriate MCDA method , 1998, Eur. J. Oper. Res..

[58]  Patrick T. Harker,et al.  The Analytic Hierarchy Process , 1989 .

[59]  Hirofumi Fukuyama,et al.  Evaluating public school district performance via DEA gain functions , 2002, J. Oper. Res. Soc..

[60]  Rozann Whitaker,et al.  Validation examples of the Analytic Hierarchy Process and Analytic Network Process , 2007, Math. Comput. Model..

[61]  T. L. Saaty.  A Scaling Method for Priorities in Hierarchical Structures , 1977 .

[62]  D.-W. Song,et al.  Weighting the competitiveness factors for container ports under conflicting interests , 2010, J. Oper. Res. Soc..

[63]  Fatemeh Zahedi,et al.  The Analytic Hierarchy Process—A Survey of the Method and its Applications , 1986 .

[64]  W. Liu,et al.  Using SSM to structure the identification of inputs and outputs in DEA , 2009, J. Oper. Res. Soc..

[65]  T. Saaty.  There is no mathematical validity for using fuzzy number crunching in the analytic hierarchy process , 2006 .