A validation scheme for intelligent and effective multiple criteria decision-making

Abstract Multiple criteria decision-making (MCDM) methods have various practical applications. Decision-makers face MCDM problems with conflicting criteria daily. Hence, MCDM methods have been developed to enable decision-makers to enhance decision quality. MCDM methods use various calculation approaches to evaluate the rank of alternatives. However, little evidence supports the consistency between the alternative chosen by the MCDM method and the decision-maker’s intuitive ideal alternative. Therefore, the objective of this study is to develop an operational validation scheme to examine and compare the effectiveness of MCDM methods. In the validation scheme, control variables include the number of alternatives, number of criteria, data set distributions, and nondominated data set options (Pareto efficient frontier or complete data set). We also add three weight distributions, namely uniform weights, rank order centroid weights, and rank sum weights, to determine the effect of weights on the MCDM methods We test linear, quadratic, Chebycheff, and prospect utility functions. In addition to the compensatory, noncompensatory, and partially compensatory utility functions, we use the prospect theory utility function. Mean absolute rank deviation and Kendall’s statistical rank test, are applied to examine the effectiveness of the methods To show the viability, this study illustrates the proposed scheme by an evaluation process of numerical comparisons among common MCDM methods including technique for order preference by similarity to ideal solution (TOPSIS), VlseKriterijumska Optimizacija I Kompromisno Resenje (VIKOR), elimination et choix traduisant la realite (ELECTRE), the piecewise linear prospect (PLP) theory method, and Analytic Hierarchy Process (AHP). Moreover, method-oriented parameter settings such as normalization methods, distance functions, VIKOR’s v, and ELECTRE’s thresholds are examined. Through the aforementioned settings, we compare the MCDM methods’ ranks with the decision-maker’s ranks by using assumed preference utility functions. The results reveal that interactive MCDM methods such as PLP and AHP outperform the others in terms of rank consistency. However, the performance of the MCDM methods is affected by the percentage of existing efficient solutions. More investigations into the applicability of the utility functions in various situations are suggested.