Consistency issues in the best worst method: Measurements and thresholds

The Best-Worst Method (BWM) uses ratios of the relative importance of criteria in pairs based on the assessment done by decision-makers. When a decision-maker provides the pairwise comparisons in BWM, checking the acceptable inconsistency, to ensure the rationality of the assessments, is an important step. Although both the original and the extended versions of BWM have proposed several consistency measurements, there are some deficiencies, including: (i) the lack of a mechanism to provide immediate feedback to the decision-maker regarding the consistency of the pairwise comparisons being provided, (ii) the inability to consider the ordinal consistency into account, and (iii) the lack of consistency thresholds to determine the reliability of the results. To deal with these problems, this study starts by proposing a cardinal consistency measurement to provide immediate feedback, called the input-based consistency measurement, after which an ordinal consistency measurement is proposed to check the coherence of the order of the results (weights) against the order of the pairwise comparisons provided by the decision-maker. Finally, a method is proposed to balance cardinal consistency ratio under ordinal-consistent and ordinal-inconsistent conditions, to determine the thresholds for the proposed and the original consistency ratios.

[1]  T. Saaty.  Fundamentals of Decision Making and Priority Theory With the Analytic Hierarchy Process , 2000 .

[2]  Youichi Iida,et al.  ORDINALITY CONSISTENCY TEST ABOUT ITEMS AND NOTATION OF A PAIRWISE COMPARISON MATRIX IN AHP , 2009 .

[3]  Gunjan Yadav,et al.  Hybrid BWM-ELECTRE-based decision framework for effective offshore outsourcing adoption: a case study , 2018, Int. J. Prod. Res..

[4]  Gang Kou,et al.  A simple method to improve the consistency ratio of the pair-wise comparison matrix in ANP , 2011, Eur. J. Oper. Res..

[5]  W. W. Koczkodaj.  A new definition of consistency of pairwise comparisons , 1993 .

[6]  P. Moran.  On the method of paired comparisons. , 1947, Biometrika.

[7]  Konrad Kulakowski,et al.  Inconsistency in the ordinal pairwise comparisons method with and without ties , 2017, Eur. J. Oper. Res..

[8]  Gabriella Marcarelli,et al.  Approximate thresholds for Salo-Hamalainen index , 2018 .

[9]  Tamás Rapcsák,et al.  On Saaty’s and Koczkodaj’s inconsistencies of pairwise comparison matrices , 2008, J. Glob. Optim..

[10]  Sen Guo,et al.  Fuzzy best-worst multi-criteria decision-making method and its applications , 2017, Knowl. Based Syst..

[11]  J. Rezaei.  Best-worst multi-criteria decision-making method , 2015 .

[12]  F. W. Irwin,et al.  An analysis of the concepts of discrimination and preference. , 1958, The American journal of psychology.

[13]  János Fülöp,et al.  On reducing inconsistency of pairwise comparison matrices below an acceptance threshold , 2013, Central Eur. J. Oper. Res..

[14]  Jafar Rezaei,et al.  A multiplicative best-worst method for multi-criteria decision making , 2019, Oper. Res. Lett..

[15]  Valdecy Pereira,et al.  Nonlinear programming applied to the reduction of inconsistency in the AHP method , 2015, Ann. Oper. Res..

[16]  John A. Keane,et al.  A heuristic method to rectify intransitive judgments in pairwise comparison matrices , 2012, Eur. J. Oper. Res..

[17]  Matteo Brunelli,et al.  A survey of inconsistency indices for pairwise comparisons , 2018, Int. J. Gen. Syst..

[18]  Ashwani Kumar,et al.  Evaluating green performance of the airports using hybrid BWM and VIKOR methodology , 2020 .

[19]  Huchang Liao,et al.  The state-of-the-art survey on integrations and applications of the best worst method in decision making: Why, what, what for and what's next? , 2019, Omega.

[20]  José María Moreno-Jiménez,et al.  The geometric consistency index: Approximated thresholds , 2003, Eur. J. Oper. Res..

[21]  Michele Fedrizzi,et al.  Axiomatic properties of inconsistency indices for pairwise comparisons , 2013, J. Oper. Res. Soc..

[22]  J. Rezaei.  Best-worst multi-criteria decision-making method: Some properties and a linear model , 2016 .

[23]  Zeshui Xu,et al.  An intuitionistic fuzzy multiplicative best-worst method for multi-criteria group decision making , 2016, Inf. Sci..

[24]  Jafar Rezaei,et al.  Quality assessment of airline baggage handling systems using SERVQUAL and BWM , 2018, Tourism Management.

[25]  José María Moreno-Jiménez,et al.  Some extensions of the precise consistency consensus matrix , 2015, Decis. Support Syst..

[26]  Konrad Kulakowski,et al.  Inconsistency indices for incomplete pairwise comparisons matrices , 2019, Int. J. Gen. Syst..

[27]  P. Harker.  Incomplete pairwise comparisons in the analytic hierarchy process , 1987 .

[28]  Giuseppe Carenini,et al.  Dealing with the Expert Inconsistency in Probability Elicitation , 2000, IEEE Trans. Knowl. Data Eng..

[29]  Enrique Herrera-Viedma,et al.  Managing incomplete preference relations in decision making: A review and future trends , 2015, Inf. Sci..

[30]  T. L. Saaty.  A Scaling Method for Priorities in Hierarchical Structures , 1977 .

[31]  Yejun Xu,et al.  The ordinal consistency of an incomplete reciprocal preference relation , 2014, Fuzzy Sets Syst..

[32]  Bice Cavallo,et al.  Weak Consistency for Ensuring Priority Vectors Reliability , 2016 .

[33]  Waldemar W. Koczkodaj,et al.  On Axiomatization of Inconsistency Indicators in Pairwise Comparisons , 2013, Int. J. Approx. Reason..

[34]  Vitaliy V. Tsyganok,et al.  The (logarithmic) least squares optimality of the arithmetic (geometric) mean of weight vectors calculated from all spanning trees for incomplete additive (multiplicative) pairwise comparison matrices , 2017, Int. J. Gen. Syst..

[35]  Salvatore Greco,et al.  The mathematical equivalence of the "spanning tree" and row geometric mean preference vectors and its implications for preference analysis , 2017, Eur. J. Oper. Res..

[36]  R. Hämäläinen,et al.  Preference programming through approximate ratio comparisons , 1995 .

[37]  Miroslaw Kwiesielewicz,et al.  Inconsistent and contradictory judgements in pairwise comparison method in the AHP , 2004, Comput. Oper. Res..

[38]  Gabriella Marcarelli,et al.  On the transitivity and consistency approximated thresholds of some consistency indices for pairwise comparison matrices , 2020, Inf. Sci..

[39]  Michele Fedrizzi,et al.  Incomplete pairwise comparison and consistency optimization , 2007, Eur. J. Oper. Res..

[40]  Michele Fedrizzi,et al.  A general formulation for some inconsistency indices of pairwise comparisons , 2018, Ann. Oper. Res..

[41]  John A. Keane,et al.  Enumerating all spanning trees for pairwise comparisons , 2012, Comput. Oper. Res..

[42]  H. Monsuur.  An intrinsic consistency threshold for reciprocal matrices , 1997 .

[43]  Peter C. Fishburn,et al.  Preference Structures and Their Numerical Representations , 1999, Theor. Comput. Sci..

[44]  Robert E. Jensen,et al.  Ordinal data AHP analysis: A proposed coefficient of consistency and a nonparametric test , 1993 .

[45]  Elizabeth Chang,et al.  ZBWM: The Z-number extension of Best Worst Method and its application for supplier development , 2018, Expert Syst. Appl..

[46]  Jacek Szybowski,et al.  On normalization of inconsistency indicators in pairwise comparisons , 2017, Int. J. Approx. Reason..

[47]  John A. Keane,et al.  Contribution of individual judgments toward inconsistency in pairwise comparisons , 2015, Eur. J. Oper. Res..

[48]  G. Crawford,et al.  A note on the analysis of subjective judgment matrices , 1985 .