Review of the Research Landscape of Multi-Criteria Evaluation and Benchmarking Processes for Many-Objective Optimization Methods: Coherent Taxonomy, Challenges and Recommended Solution

Evaluation and benchmarking of many-objective optimization (MaOO) methods are complicated. The rapid development of new optimization algorithms for solving problems with many objectives has increas...

[1]  Yaochu Jin,et al.  A radial space division based evolutionary algorithm for many-objective optimization , 2017, Appl. Soft Comput..

[2]  Xin Yao,et al.  Corner Sort for Pareto-Based Many-Objective Optimization , 2014, IEEE Transactions on Cybernetics.

[3]  E. Stanley Lee,et al.  An extension of TOPSIS for group decision making , 2007, Math. Comput. Model..

[4]  Aurora Trinidad Ramirez Pozo,et al.  Transfer weight functions for injecting problem information in the multi-objective CMA-ES , 2016, Memetic Computing.

[5]  Lihong Xu,et al.  Many-objective fuzzy centroids clustering algorithm for categorical data , 2018, Expert Syst. Appl..

[6]  Weiwei Zhang,et al.  Cooperative Differential Evolution With Multiple Populations for Multiobjective Optimization , 2016, IEEE Transactions on Cybernetics.

[7]  Gwo-Hshiung Tzeng,et al.  Extended VIKOR method in comparison with outranking methods , 2007, Eur. J. Oper. Res..

[8]  Ankur Sinha,et al.  Convex preference cone-based approach for many objective optimization problems , 2018, Comput. Oper. Res..

[9]  Markus Olhofer,et al.  Test Problems for Large-Scale Multiobjective and Many-Objective Optimization , 2017, IEEE Transactions on Cybernetics.

[10]  B. B. Zaidan,et al.  Smart Home-based IoT for Real-time and Secure Remote Health Monitoring of Triage and Priority System using Body Sensors: Multi-driven Systematic Review , 2019, Journal of Medical Systems.

[11]  B. B. Zaidan,et al.  Towards on Develop a Framework for the Evaluation and Benchmarking of Skin Detectors Based on Artificial Intelligent Models Using Multi-Criteria Decision-Making Techniques , 2017, Int. J. Pattern Recognit. Artif. Intell..

[12]  A. A. Zaidan,et al.  Multi-Criteria Evaluation and Benchmarking for Young Learners’ English Language Mobile Applications in Terms of LSRW Skills , 2019, IEEE Access.

[13]  Slavka Bodjanova,et al.  Median alpha-levels of a fuzzy number , 2006, Fuzzy Sets Syst..

[14]  Amelia Ritahani Ismail,et al.  A Review of Data Analysis for Early-Childhood Period: Taxonomy, Motivations, Challenges, Recommendation, and Methodological Aspects , 2019, IEEE Access.

[15]  A. A. Zaidan,et al.  Mobile Patient Monitoring Systems from a Benchmarking Aspect: Challenges, Open Issues and Recommended Solutions , 2019, Journal of Medical Systems.

[16]  Charanjit Kaur Swaran Singh,et al.  Assessment and Ranking Framework for the English Skills of Pre-Service Teachers Based on Fuzzy Delphi and TOPSIS Methods , 2019, IEEE Access.

[17]  Haoran Sun,et al.  A diversity indicator based on reference vectors for many-objective optimization , 2018, Inf. Sci..

[18]  Alain Bernard,et al.  Embedding Multi-Attribute Decision Making into Evolutionary Optimization to Solve the Many-Objective Combinatorial Optimization Problems , 2016 .

[19]  Bin Zhang,et al.  Decomposition-based sub-problem optimal solution updating direction-guided evolutionary many-objective algorithm , 2018, Inf. Sci..

[20]  Gang Kou,et al.  Evaluation of feature selection methods for text classification with small datasets using multiple criteria decision-making methods , 2020, Appl. Soft Comput..

[21]  Jitender Kumar Chhabra,et al.  FP-ABC: Fuzzy-Pareto dominance driven artificial bee colony algorithm for many-objective software module clustering , 2018, Comput. Lang. Syst. Struct..

[22]  Chao Wang,et al.  An improved NSGA-III algorithm based on objective space decomposition for many-objective optimization , 2017, Soft Comput..

[23]  Felipe Campelo,et al.  Preference-guided evolutionary algorithms for many-objective optimization , 2016, Inf. Sci..

[24]  B. B. Zaidan,et al.  Real-Time Medical Systems Based on Human Biometric Steganography: a Systematic Review , 2018, Journal of Medical Systems.

[25]  F. M. Jumaah,et al.  Decision-making solution based multi-measurement design parameter for optimization of GPS receiver tracking channels in static and dynamic real-time positioning multipath environment , 2018 .

[26]  Okkes Tolga Altinöz,et al.  Multiobjective Hooke-Jeeves algorithm with a stochastic Newton-Raphson-like step-size method , 2019, Expert Syst. Appl..

[27]  Naheed Anjum Arafat,et al.  Evolutionary algorithm using adaptive fuzzy dominance and reference point for many-objective optimization , 2019, Swarm Evol. Comput..

[28]  B. B. Zaidan,et al.  Multi-criteria analysis for OS-EMR software selection problem: A comparative study , 2015, Decis. Support Syst..

[29]  Elizabeth Chang,et al.  ZBWM: The Z-number extension of Best Worst Method and its application for supplier development , 2018, Expert Syst. Appl..

[30]  Wei Zheng,et al.  An Improved MOEA/D with Optimal DE Schemes for Many-Objective Optimization Problems , 2017, Algorithms.

[31]  Xin Yao,et al.  Stochastic Ranking Algorithm for Many-Objective Optimization Based on Multiple Indicators , 2016, IEEE Transactions on Evolutionary Computation.

[32]  Yanli Yin,et al.  User-oriented many-objective cloud workflow scheduling based on an improved knee point driven evolutionary algorithm , 2017, Knowl. Based Syst..

[33]  Monalisa Pal,et al.  NAEMO: Neighborhood-sensitive archived evolutionary many-objective optimization algorithm , 2019, Swarm Evol. Comput..

[34]  Leo L. Pipino,et al.  A pilot study of fuzzy set modification of Delphi , 1985 .

[35]  Xin Yao,et al.  Many-Objective Evolutionary Algorithms , 2015, ACM Comput. Surv..

[36]  Yi Peng,et al.  A Group Decision Making Model for Integrating Heterogeneous Information , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[37]  Michael G. Epitropakis,et al.  Progressive preference articulation for decision making in multi-objective optimisation problems , 2017, Integr. Comput. Aided Eng..

[38]  Gwo-Hshiung Tzeng,et al.  Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS , 2004, Eur. J. Oper. Res..

[39]  Xiaoyan Sun,et al.  Set-based many-objective optimization guided by a preferred region , 2017, Neurocomputing.

[40]  Ching-Hsue Cheng,et al.  Evaluating the best main battle tank using fuzzy decision theory with linguistic criteria evaluation , 2002, Eur. J. Oper. Res..

[41]  Wang Hu,et al.  Many-Objective Particle Swarm Optimization Using Two-Stage Strategy and Parallel Cell Coordinate System , 2017, IEEE Transactions on Cybernetics.

[42]  Heike Trautmann,et al.  2 Indicator-Based Multiobjective Search , 2015, Evolutionary Computation.

[43]  Desmond Greer,et al.  Using a many-objective approach to investigate automated refactoring , 2019, Inf. Softw. Technol..

[44]  Chia-Wei Tang,et al.  Obtaining a picture of undergraduate education quality: a voice from inside the university , 2010 .

[45]  A. A. Zaidan,et al.  Comparative study on the evaluation and benchmarking information hiding approaches based multi-measurement analysis using TOPSIS method with different normalisation, separation and context techniques , 2018 .

[46]  Xia Li,et al.  A decomposition-based multi-objective evolutionary algorithm with quality indicator , 2018, Swarm Evol. Comput..

[47]  Yiu-Ming Cheung,et al.  Self-Organizing Map-Based Weight Design for Decomposition-Based Many-Objective Evolutionary Algorithm , 2018, IEEE Transactions on Evolutionary Computation.

[48]  Yuping Wang,et al.  An improved $${\alpha }$$α-dominance strategy for many-objective optimization problems , 2016, Soft Comput..

[49]  Bin Chen,et al.  A many-objective optimization of industrial environmental management using NSGA-III: A case of China’s iron and steel industry , 2019, Applied Energy.

[50]  Gexiang Zhang,et al.  A Many-Objective Evolutionary Algorithm With Enhanced Mating and Environmental Selections , 2015, IEEE Transactions on Evolutionary Computation.

[51]  Eduardo Segredo,et al.  Impact of selection methods on the diversity of many-objective Pareto set approximations , 2017, KES.

[52]  Benjamín Barán,et al.  Many-Objective Virtual Machine Placement , 2017, Journal of Grid Computing.

[53]  Hisao Ishibuchi,et al.  Pareto Fronts of Many-Objective Degenerate Test Problems , 2016, IEEE Transactions on Evolutionary Computation.

[54]  Sebastián Ventura,et al.  JCLEC-MO: A Java suite for solving many-objective optimization engineering problems , 2019, Eng. Appl. Artif. Intell..

[55]  A. Ishikawa,et al.  The Max-Min Delphi method and fuzzy Delphi method via fuzzy integration , 1993 .

[56]  Muhammet Gulź,et al.  A state of the art literature review of VIKOR and its fuzzy extensions on applications , 2016 .

[57]  P. H. Huang,et al.  A non-linear non-weight method for multi-criteria decision making , 2017, Ann. Oper. Res..

[58]  Lei Huang,et al.  A many-objective evolutionary algorithm with epsilon-indicator direction vector , 2019, Appl. Soft Comput..

[59]  Hiroyuki Sato,et al.  Adaptive Control of Dominance Area of Solutions in Evolutionary Many-Objective Optimization , 2015, New Math. Nat. Comput..

[60]  Edmundas Kazimieras Zavadskas,et al.  Multi-Attribute Decision-Making Model by Applying Grey Numbers , 2009, Informatica.

[61]  Zhang Yi,et al.  Reference line-based Estimation of Distribution Algorithm for many-objective optimization , 2017, Knowl. Based Syst..

[62]  Prashant K. Jamwal,et al.  Evolutionary Optimization Using Equitable Fuzzy Sorting Genetic Algorithm (EFSGA) , 2019, IEEE Access.

[63]  B. B. Zaidan,et al.  Real-Time Remote-Health Monitoring Systems: a Review on Patients Prioritisation for Multiple-Chronic Diseases, Taxonomy Analysis, Concerns and Solution Procedure , 2019, Journal of Medical Systems.

[64]  Po-Chien Chang,et al.  Fuzzy Delphi method for evaluating hydrogen production technologies , 2011 .

[65]  B. B. Zaidan,et al.  Systematic Review of Real-time Remote Health Monitoring System in Triage and Priority-Based Sensor Technology: Taxonomy, Open Challenges, Motivation and Recommendations , 2018, Journal of Medical Systems.

[66]  Carlo Cavallini,et al.  Integral aided method for material selection based on quality function deployment and comprehensive VIKOR algorithm , 2013 .

[67]  Ching-Lai Hwang,et al.  Multiple Attribute Decision Making: Methods and Applications - A State-of-the-Art Survey , 1981, Lecture Notes in Economics and Mathematical Systems.

[68]  Sergio Segura,et al.  SIP: Optimal Product Selection from Feature Models Using Many-Objective Evolutionary Optimization , 2016, ACM Trans. Softw. Eng. Methodol..

[69]  Xin Yao,et al.  Nadir point estimation for many-objective optimization problems based on emphasized critical regions , 2017, Soft Comput..

[70]  B. B. Zaidan,et al.  Multi-Criteria Evaluation and Benchmarking for Active Queue Management Methods: Open Issues, Challenges and Recommended Pathway Solutions , 2019, Int. J. Inf. Technol. Decis. Mak..

[71]  Gül E. Okudan,et al.  Fuzzy AHP and utility theory based patient sorting in emergency departments , 2010 .

[72]  Eduardo José Solteiro Pires,et al.  Many-objective optimization with corner-based search , 2015, Memetic Comput..

[73]  Ye Tian,et al.  An Indicator-Based Multiobjective Evolutionary Algorithm With Reference Point Adaptation for Better Versatility , 2018, IEEE Transactions on Evolutionary Computation.

[74]  Yang Gao,et al.  A new hypervolume-based differential evolution algorithm for many-objective optimization , 2017, RAIRO Oper. Res..

[75]  Ferrante Neri,et al.  A fast hypervolume driven selection mechanism for many-objective optimisation problems , 2017, Swarm Evol. Comput..

[76]  Qingfu Zhang,et al.  Entropy-Based Termination Criterion for Multiobjective Evolutionary Algorithms , 2016, IEEE Transactions on Evolutionary Computation.

[77]  Tapabrata Ray,et al.  A Novel Decomposition Based Evolutionary Algorithm for Engineering Design Optimization , 2017 .

[78]  Ali Hamzeh,et al.  A coevolutionary approach to many objective optimization based on a novel ranking method , 2016, Intell. Data Anal..

[79]  Xin Yao,et al.  A New Dominance Relation-Based Evolutionary Algorithm for Many-Objective Optimization , 2016, IEEE Transactions on Evolutionary Computation.

[80]  Yalan Zhou,et al.  Ensemble of many-objective evolutionary algorithms for many-objective problems , 2015, Soft Computing.

[81]  Robert M Kalin,et al.  Optimum socio-environmental flows approach for reservoir operation strategy using many-objectives evolutionary optimization algorithm. , 2019, The Science of the total environment.

[82]  C. Powell.  The Delphi technique: myths and realities. , 2003, Journal of advanced nursing.

[83]  A. A. Zaidan,et al.  An evaluation and selection problems of OSS-LMS packages , 2016, SpringerPlus.

[84]  Kalyanmoy Deb,et al.  A Taxonomy for Metamodeling Frameworks for Evolutionary Multiobjective Optimization , 2019, IEEE Transactions on Evolutionary Computation.

[85]  Ye Tian,et al.  Approximate non-dominated sorting for evolutionary many-objective optimization , 2016, Inf. Sci..

[86]  Yi Peng,et al.  Soft consensus cost models for group decision making and economic interpretations , 2019, Eur. J. Oper. Res..

[87]  A. A. Zaidan,et al.  Comprehensive Insights Into the Criteria of Student Performance in Various Educational Domains , 2018, IEEE Access.

[88]  Hisao Ishibuchi,et al.  Guest Editorial Evolutionary Many-Objective Optimization , 2018, IEEE Trans. Evol. Comput..

[89]  Qingfu Zhang,et al.  Decomposition-Based-Sorting and Angle-Based-Selection for Evolutionary Multiobjective and Many-Objective Optimization , 2017, IEEE Transactions on Cybernetics.

[90]  Jun Zhang,et al.  Particle Swarm Optimization With a Balanceable Fitness Estimation for Many-Objective Optimization Problems , 2018, IEEE Transactions on Evolutionary Computation.

[91]  Chuang Liu,et al.  A membrane algorithm based on chemical reaction optimization for many-objective optimization problems , 2019, Knowl. Based Syst..

[92]  Gwo-Jen Hwang,et al.  A Delphi-based approach to developing expert systems with the cooperation of multiple experts , 2007, Expert Systems with Applications.

[93]  Licheng Jiao,et al.  An r-dominance-based preference multi-objective optimization for many-objective optimization , 2017, Soft Comput..

[94]  Songfeng Lu,et al.  Many-objectives multilevel thresholding image segmentation using Knee Evolutionary Algorithm , 2019, Expert Syst. Appl..

[95]  Yi Peng,et al.  Evaluation of clustering algorithms for financial risk analysis using MCDM methods , 2014, Inf. Sci..

[96]  Shengxiang Yang,et al.  A many-objective evolutionary algorithm based on rotated grid , 2018, Appl. Soft Comput..

[97]  Gwo-Hshiung Tzeng,et al.  A modified VIKOR multiple-criteria decision method for improving domestic airlines service quality , 2011 .

[98]  Xin Yao,et al.  Multiline Distance Minimization: A Visualized Many-Objective Test Problem Suite , 2018, IEEE Transactions on Evolutionary Computation.

[99]  B. B. Zaidan,et al.  Real-Time Fault-Tolerant mHealth System: Comprehensive Review of Healthcare Services, Opens Issues, Challenges and Methodological Aspects , 2018, Journal of Medical Systems.

[100]  A. A. Zaidan,et al.  Mobile-Based Patient Monitoring Systems: A Prioritisation Framework Using Multi-Criteria Decision-Making Techniques , 2019, Journal of Medical Systems.

[101]  Miss Laiha Mat Kiah,et al.  Comprehensive review and analysis of anti-malware apps for smartphones , 2019, Telecommunication Systems.

[102]  Mohd Alauddin Mohd Ali,et al.  Comparison of adaptive neuro-fuzzy inference system and artificial neutral networks model to categorize patients in the emergency department , 2013, SpringerPlus.

[103]  Hua Xu,et al.  Objective Reduction in Many-Objective Optimization: Evolutionary Multiobjective Approaches and Comprehensive Analysis , 2018, IEEE Transactions on Evolutionary Computation.

[104]  Adriana Menchaca-Mendez,et al.  An alternative hypervolume-based selection mechanism for multi-objective evolutionary algorithms , 2017, Soft Comput..

[105]  Bansi D. Raja,et al.  A comparative performance evaluation of the reversed Brayton cycle operated heat pump based on thermo-ecological criteria through many and multi objective approaches , 2019, Energy Conversion and Management.

[106]  Kalyanmoy Deb,et al.  A Unified Evolutionary Optimization Procedure for Single, Multiple, and Many Objectives , 2016, IEEE Transactions on Evolutionary Computation.

[107]  Hristos Karahalios,et al.  The application of the AHP-TOPSIS for evaluating ballast water treatment systems by ship operators , 2017 .

[108]  Toon De Pessemier,et al.  Energy- and labor-aware flexible job shop scheduling under dynamic electricity pricing: A many-objective optimization investigation , 2019, Journal of Cleaner Production.

[109]  Erik D. Goodman,et al.  A novel two-archive matching-based algorithm for multi- and many-objective optimization , 2019, Inf. Sci..

[110]  Teresa Bernarda Ludermir,et al.  Many Objective Particle Swarm Optimization , 2016, Inf. Sci..

[111]  K. Secnik,et al.  The association between diabetes related medical costs and glycemic control: A retrospective analysis , 2006, Cost effectiveness and resource allocation : C/E.

[112]  A. A. Zaidan,et al.  A methodology for football players selection problem based on multi-measurements criteria analysis , 2017 .

[113]  Gary G. Yen,et al.  Visualization and Performance Metric in Many-Objective Optimization , 2016, IEEE Transactions on Evolutionary Computation.

[114]  A. Taleb-Ahmed,et al.  Fast Solutions Enhancing using a Copula-based EDA and SVM for many-objective problems , 2016 .

[115]  Theodor J. Stewart,et al.  Multiple Criteria Decision Analysis , 2001 .

[116]  Gina Maira Barbosa de Oliveira,et al.  MEANDS: A Many-objective Evolutionary Algorithm based on Non-dominated Decomposed Sets applied to multicast routing , 2018, Appl. Soft Comput..

[117]  Xiaoyan Sun,et al.  Many-objective evolutionary optimization based on reference points , 2017, Appl. Soft Comput..

[118]  Gary G. Yen,et al.  Many-Objective Evolutionary Algorithm: Objective Space Reduction and Diversity Improvement , 2016, IEEE Transactions on Evolutionary Computation.

[119]  Dunwei Gong,et al.  A Set-Based Genetic Algorithm for Interval Many-Objective Optimization Problems , 2018, IEEE Transactions on Evolutionary Computation.

[120]  Tao Zhang,et al.  Localized Weighted Sum Method for Many-Objective Optimization , 2018, IEEE Transactions on Evolutionary Computation.

[121]  B. B. Zaidan,et al.  Multiclass Benchmarking Framework for Automated Acute Leukaemia Detection and Classification Based on BWM and Group-VIKOR , 2019, Journal of Medical Systems.

[122]  B. B. Zaidan,et al.  A survey on communication components for IoT-based technologies in smart homes , 2018, Telecommunication Systems.

[123]  İhsan Kaya,et al.  Use of MCDM techniques for energy policy and decision‐making problems: A review , 2018 .

[124]  Maarten J. IJzerman,et al.  Multiple Criteria Decision Analysis for Health Care Decision Making--An Introduction: Report 1 of the ISPOR MCDA Emerging Good Practices Task Force. , 2016, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[125]  Jing Sun,et al.  A mutation operator guided by preferred regions for set-based many-objective evolutionary optimization , 2017 .

[126]  Guangming Dai,et al.  Indicator and reference points co-guided evolutionary algorithm for many-objective optimization problems , 2018, Knowl. Based Syst..

[127]  Aduwati Sali,et al.  Multi-complex attributes analysis for optimum GPS baseband receiver tracking channels selection , 2017, 2017 4th International Conference on Control, Decision and Information Technologies (CoDIT).

[128]  Yi Peng,et al.  Evaluation of Classification Algorithms Using MCDM and Rank Correlation , 2012, Int. J. Inf. Technol. Decis. Mak..

[129]  Aviad Shapira,et al.  AHP-based analysis of the risk potential of safety incidents: Case study of cranes in the construction industry , 2017 .

[130]  B. B. Zaidan,et al.  Blockchain authentication of network applications: Taxonomy, classification, capabilities, open challenges, motivations, recommendations and future directions , 2019, Comput. Stand. Interfaces.

[131]  Patrick M. Reed,et al.  Reducing regional drought vulnerabilities and multi-city robustness conflicts using many-objective optimization under deep uncertainty , 2015 .

[132]  C. Duffield.  The Delphi technique: a comparison of results obtained using two expert panels. , 1993, International journal of nursing studies.

[133]  Yi Liang,et al.  Objective reduction particle swarm optimizer based on maximal information coefficient for many-objective problems , 2017, Neurocomputing.

[134]  B. B. Zaidan,et al.  Based on Real Time Remote Health Monitoring Systems: A New Approach for Prioritization “Large Scales Data” Patients with Chronic Heart Diseases Using Body Sensors and Communication Technology , 2018, Journal of Medical Systems.

[135]  Bo Zhang,et al.  Balancing Convergence and Diversity in Decomposition-Based Many-Objective Optimizers , 2016, IEEE Transactions on Evolutionary Computation.

[136]  Wang Yu-ping,et al.  A new uniform evolutionary algorithm based on decomposition and CDAS for many-objective optimization , 2015 .

[137]  Yang Chen,et al.  Pairwise comparison matrix in multiple criteria decision making , 2016 .

[138]  A. A. Zaidan,et al.  Finger Vein Biometrics: Taxonomy Analysis, Open Challenges, Future Directions, and Recommended Solution for Decentralised Network Architectures , 2020, IEEE Access.

[139]  Bernhard Sendhoff,et al.  Preference representation using Gaussian functions on a hyperplane in evolutionary multi-objective optimization , 2015, Soft Computing.

[140]  Linqiang Pan,et al.  Switching ripple suppressor design of the grid-connected inverters: A perspective of many-objective optimization with constraints handling , 2019, Swarm Evol. Comput..

[141]  J. Rezaei.  Best-worst multi-criteria decision-making method , 2015 .

[142]  Holger R. Maier,et al.  Many-objective portfolio optimization approach for stormwater management project selection encouraging decision maker buy-in , 2019, Environ. Model. Softw..

[143]  B. B. Zaidan,et al.  Software and Hardware FPGA-Based Digital Watermarking and Steganography Approaches: Toward New Methodology for Evaluation and Benchmarking Using Multi-Criteria Decision-Making Techniques , 2017, J. Circuits Syst. Comput..

[144]  S. M. Sapuan,et al.  A comprehensive VIKOR method for material selection , 2011, Materials & Design.

[145]  Slim Bechikh,et al.  A New Decomposition-Based NSGA-II for Many-Objective Optimization , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[146]  Himanshu Gupta,et al.  Evaluating service quality of airline industry using hybrid best worst method and VIKOR , 2017 .

[147]  Seyed Amin Seyed Haeri,et al.  Airports evaluation and ranking model using Taguchi loss function, best-worst method and VIKOR technique , 2017 .

[148]  Alex F. Mills,et al.  A simple yet effective decision support policy for mass-casualty triage , 2016, Eur. J. Oper. Res..

[149]  Ye Tian,et al.  Effectiveness and efficiency of non-dominated sorting for evolutionary multi- and many-objective optimization , 2017, Complex & Intelligent Systems.

[150]  Beatrice M. Ombuki-Berman,et al.  A Scalability Study of Many-Objective Optimization Algorithms , 2018, IEEE Transactions on Evolutionary Computation.

[151]  Lóránt Tavasszy,et al.  Evaluation of the external forces affecting the sustainability of oil and gas supply chain using Best Worst Method , 2017 .

[152]  Sen Guo,et al.  Fuzzy best-worst multi-criteria decision-making method and its applications , 2017, Knowl. Based Syst..

[153]  Hisao Ishibuchi,et al.  Benchmarking Multi- and Many-Objective Evolutionary Algorithms Under Two Optimization Scenarios , 2017, IEEE Access.

[154]  Shuo-Yan Chou,et al.  A fuzzy simple additive weighting system under group decision-making for facility location selection with objective/subjective attributes , 2008, Eur. J. Oper. Res..

[155]  Jafar Rezaei,et al.  Evaluating firms' R&D performance using best worst method. , 2018, Evaluation and program planning.

[156]  Tapabrata Ray,et al.  Bridging the Gap: Many-Objective Optimization and Informed Decision-Making , 2017, IEEE Transactions on Evolutionary Computation.

[157]  Walid Serrai,et al.  Towards an efficient and a more accurate web service selection using MCDM methods , 2017, J. Comput. Sci..

[158]  Gary G. Yen,et al.  Many-Objective Evolutionary Algorithms Based on Coordinated Selection Strategy , 2017, IEEE Transactions on Evolutionary Computation.

[159]  Carlos A. Coello Coello,et al.  A new indicator-based many-objective ant colony optimizer for continuous search spaces , 2017, Swarm Intelligence.

[160]  Zhaolu Guo,et al.  Many-objective E-dominance dynamical evolutionary algorithm based on adaptive grid , 2018, Soft Comput..

[161]  Georges Adunlin,et al.  Application of multicriteria decision analysis in health care: a systematic review and bibliometric analysis , 2015, Health expectations : an international journal of public participation in health care and health policy.

[162]  Jinhua Zheng,et al.  Many-objective optimization based on information separation and neighbor punishment selection , 2015, Soft Comput..

[163]  Duc Truong Pham,et al.  Evolutionary many-objective optimization for mixed-model disassembly line balancing with multi-robotic workstations , 2019, Eur. J. Oper. Res..

[164]  Weili Wang,et al.  A performance-driven multi-algorithm selection strategy for energy consumption optimization of sea-rail intermodal transportation , 2019, Swarm Evol. Comput..

[165]  Tapan Kumar Roy,et al.  WEIGHTED INTUITIONISTIC FUZZY DELPHI METHOD , 2013 .

[166]  Changhe Li,et al.  Many-objective optimization with dynamic constraint handling for constrained optimization problems , 2017, Soft Comput..

[167]  Zhong Ming,et al.  An improved NSGA-III algorithm for feature selection used in intrusion detection , 2017, Knowl. Based Syst..

[168]  B. B. Zaidan,et al.  Systematic Review of an Automated Multiclass Detection and Classification System for Acute Leukaemia in Terms of Evaluation and Benchmarking, Open Challenges, Issues and Methodological Aspects , 2018, Journal of Medical Systems.

[169]  Xin Li,et al.  Generational Distance Indicator-Based Evolutionary Algorithm With an Improved Niching Method for Many-Objective Optimization Problems , 2019, IEEE Access.

[170]  J. Murry,et al.  Delphi: A Versatile Methodology for Conducting Qualitative Research , 2017 .

[171]  Qiuzhen Lin,et al.  A hybridized angle-encouragement-based decomposition approach for many-objective optimization problems , 2019, Appl. Soft Comput..

[172]  R. Baltussen,et al.  Priority setting of health interventions: the need for multi-criteria decision analysis , 2006, Cost effectiveness and resource allocation : C/E.

[173]  Xinye Cai,et al.  A Decomposition-Based Many-Objective Evolutionary Algorithm With Two Types of Adjustments for Direction Vectors , 2018, IEEE Transactions on Cybernetics.

[174]  Haopeng Zhang,et al.  Many objective cooperative bat searching algorithm , 2019, Appl. Soft Comput..

[175]  Shengxiang Yang,et al.  A Strength Pareto Evolutionary Algorithm Based on Reference Direction for Multiobjective and Many-Objective Optimization , 2017, IEEE Transactions on Evolutionary Computation.

[176]  Frederico G. Guimarães,et al.  A new visualization method in many-objective optimization with chord diagram and angular mapping , 2017, Knowl. Based Syst..

[177]  Chao Wang,et al.  An improved NSGA-III algorithm based on elimination operator for many-objective optimization , 2017, Memetic Comput..

[178]  Adel Hatami-Marbini,et al.  A group AHP-TOPSIS framework for human spaceflight mission planning at NASA , 2011, Expert Syst. Appl..

[179]  Kaisa Miettinen,et al.  A Surrogate-Assisted Reference Vector Guided Evolutionary Algorithm for Computationally Expensive Many-Objective Optimization , 2018, IEEE Transactions on Evolutionary Computation.

[180]  B. B. Zaidan,et al.  Evaluation and selection of open-source EMR software packages based on integrated AHP and TOPSIS , 2015, J. Biomed. Informatics.

[181]  K. I. Mohammed,et al.  Based Multiple Heterogeneous Wearable Sensors: A Smart Real-Time Health Monitoring Structured for Hospitals Distributor , 2019, IEEE Access.

[182]  Ali Haghighi,et al.  Stability analysis of gravity dams under uncertainty using the fuzzy sets theory and a many-objective GA , 2016, J. Intell. Fuzzy Syst..

[183]  Hani Hagras,et al.  iPatch: A Many-Objective Type-2 Fuzzy Logic System for Field Workforce Optimization , 2019, IEEE Transactions on Fuzzy Systems.

[184]  Josef Jablonsky,et al.  MS Excel based Software Support Tools for Decision Problems with Multiple Criteria , 2014 .

[185]  Gwo-Hshiung Tzeng,et al.  A new hybrid MCDM model combining DANP with VIKOR to improve e-store business , 2013, Knowl. Based Syst..

[186]  Xiaobing Yu,et al.  Evaluation of Many-Objective Evolutionary Algorithms by Hesitant Fuzzy Linguistic Term Set and Majority Operator , 2018, Int. J. Fuzzy Syst..

[187]  Aurora Trinidad Ramirez Pozo,et al.  C-Multi: A competent multi-swarm approach for many-objective problems , 2016, Neurocomputing.

[188]  Xin Yao,et al.  Diversity Assessment in Many-Objective Optimization , 2017, IEEE Transactions on Cybernetics.

[189]  B. B. Zaidan,et al.  Real-Time Remote Health-Monitoring Systems in a Medical Centre: A Review of the Provision of Healthcare Services-Based Body Sensor Information, Open Challenges and Methodological Aspects , 2018, Journal of Medical Systems.

[190]  Lei Cai,et al.  Two-archive method for aggregation-based many-objective optimization , 2018, Inf. Sci..

[191]  Fei Li,et al.  A two-stage R2 indicator based evolutionary algorithm for many-objective optimization , 2018, Appl. Soft Comput..

[192]  Morteza Yazdani,et al.  A state-of the-art survey of TOPSIS applications , 2012, Expert Syst. Appl..

[193]  Tapabrata Ray,et al.  An Enhanced Decomposition-Based Evolutionary Algorithm With Adaptive Reference Vectors , 2018, IEEE Transactions on Cybernetics.

[194]  Peter J. Fleming,et al.  Preference-Inspired Coevolutionary Algorithms for Many-Objective Optimization , 2013, IEEE Transactions on Evolutionary Computation.

[195]  Himanshu Gupta,et al.  Supplier selection among SMEs on the basis of their green innovation ability using BWM and fuzzy TOPSIS , 2017 .

[196]  Marouane Kessentini,et al.  Model Transformation Modularization as a Many-Objective Optimization Problem , 2017, IEEE Transactions on Software Engineering.

[197]  Yuren Zhou,et al.  A Vector Angle-Based Evolutionary Algorithm for Unconstrained Many-Objective Optimization , 2017, IEEE Transactions on Evolutionary Computation.

[198]  Erik D. Goodman,et al.  Triple Bottomline Many‐Objective‐Based Decision Making for a Land Use Management Problem , 2015 .

[199]  Antonio López Jaimes,et al.  An investigation into many-objective optimization on combinatorial problems: Analyzing the pickup and delivery problem , 2018, Swarm Evol. Comput..

[200]  Zexuan Zhu,et al.  Two new reference vector adaptation strategies for many-objective evolutionary algorithms , 2019, Inf. Sci..

[201]  Bernhard Sendhoff,et al.  A Reference Vector Guided Evolutionary Algorithm for Many-Objective Optimization , 2016, IEEE Transactions on Evolutionary Computation.

[202]  Qingfu Zhang,et al.  A Grid Weighted Sum Pareto Local Search for Combinatorial Multi and Many-Objective Optimization , 2019, IEEE Transactions on Cybernetics.

[203]  B. B. Zaidan,et al.  Real-Time Remote Health Monitoring Systems Using Body Sensor Information and Finger Vein Biometric Verification: A Multi-Layer Systematic Review , 2018, Journal of Medical Systems.

[204]  Xiaoyan Sun,et al.  Indicator-based set evolution particle swarm optimization for many-objective problems , 2016, Soft Comput..

[205]  Yuren Zhou,et al.  An angle based constrained many-objective evolutionary algorithm , 2017, Applied Intelligence.

[206]  A. A. Zaidan,et al.  A new digital watermarking evaluation and benchmarking methodology using an external group of evaluators and multi‐criteria analysis based on ‘large‐scale data’ , 2017, Softw. Pract. Exp..

[207]  Ricardo H. C. Takahashi,et al.  On the Performance Degradation of Dominance-Based Evolutionary Algorithms in Many-Objective Optimization , 2018, IEEE Transactions on Evolutionary Computation.

[208]  Mehmet Altinoz,et al.  Systematic Initialization Approaches for Portfolio Optimization Problems , 2019, IEEE Access.

[209]  Yuping Wang,et al.  An objective reduction algorithm using representative Pareto solution search for many-objective optimization problems , 2016, Soft Comput..

[210]  Yi Xiang,et al.  A many-objective evolutionary algorithm based on a projection-assisted intra-family election , 2017, Appl. Soft Comput..

[211]  Cai Dai,et al.  A new decomposition based evolutionary algorithm with uniform designs for many-objective optimization , 2015, Appl. Soft Comput..

[212]  Markus Olhofer,et al.  Evolutionary Many-Objective Optimization of Hybrid Electric Vehicle Control: From General Optimization to Preference Articulation , 2017, IEEE Transactions on Emerging Topics in Computational Intelligence.

[213]  Xin Liu,et al.  Distributed Parallel Particle Swarm Optimization for Multi-Objective and Many-Objective Large-Scale Optimization , 2017, IEEE Access.

[214]  B. B. Zaidan,et al.  Technique for order performance by similarity to ideal solution for solving complex situations in multi-criteria optimization of the tracking channels of GPS baseband telecommunication receivers , 2017, Telecommunication Systems.

[215]  B. B. Zaidan,et al.  Based Real Time Remote Health Monitoring Systems: A Review on Patients Prioritization and Related "Big Data" Using Body Sensors information and Communication Technology , 2018, Journal of Medical Systems.

[216]  Kazuyuki Murase,et al.  Evolutionary Path Control Strategy for Solving Many-Objective Optimization Problem , 2015, IEEE Transactions on Cybernetics.

[217]  Nitin Muttil,et al.  Using the Analytic Hierarchy Process to identify parameter weights for developing a water quality index , 2017 .

[218]  Yutao Qi,et al.  User-preference based decomposition in MOEA/D without using an ideal point , 2019, Swarm Evol. Comput..

[219]  Mehmet Alper Sofuoğlu,et al.  A Novel Hybrid Multi Criteria Decision Making Model: Application to Turning Operations , 2017 .

[220]  Xuyan Tu,et al.  Oriented multi-mutation strategy in a many-objective evolutionary algorithm , 2019, Inf. Sci..

[221]  Xin Yao,et al.  A clustering-ranking method for many-objective optimization , 2015, Appl. Soft Comput..

[222]  A. A. Zaidan,et al.  A New Approach based on Multi-Dimensional Evaluation and Benchmarking for Data Hiding Techniques , 2017 .

[223]  Y. N. KOU,et al.  Many-objective optimization for coordinated operation of integrated electricity and gas network , 2017 .

[224]  Mohamed Wiem Mkaouer,et al.  On the use of many quality attributes for software refactoring: a many-objective search-based software engineering approach , 2016, Empirical Software Engineering.

[225]  Hisao Ishibuchi,et al.  Performance of Decomposition-Based Many-Objective Algorithms Strongly Depends on Pareto Front Shapes , 2017, IEEE Transactions on Evolutionary Computation.

[226]  Yuren Zhou,et al.  A Decomposition-Based Many-Objective Artificial Bee Colony Algorithm , 2019, IEEE Transactions on Cybernetics.

[227]  Folke Björk,et al.  Owner preferences regarding renovation measures – the demonstration of using multi-criteria decision making , 2011 .

[228]  Rodrigo C. P. Silva,et al.  Implementation of Iron Loss Model on Graphic Processing Units , 2016, IEEE Transactions on Magnetics.

[229]  Sergio Segura,et al.  Evolutionary composition of QoS-aware web services: A many-objective perspective , 2017, Expert Syst. Appl..

[230]  Kiyoshi Tanaka,et al.  A Review of Features and Limitations of Existing Scalable Multiobjective Test Suites , 2019, IEEE Transactions on Evolutionary Computation.

[231]  Zeng Tao,et al.  An improved NSGA-III integrating adaptive elimination strategy to solution of many-objective optimal power flow problems , 2019, Energy.

[232]  J. Rezaei.  Best-worst multi-criteria decision-making method: Some properties and a linear model , 2016 .

[233]  Xin Yao,et al.  A benchmark test suite for evolutionary many-objective optimization , 2017, Complex & Intelligent Systems.

[234]  Jinhua Zheng,et al.  Binary search based boundary elimination selection in many-objective evolutionary optimization , 2017, Appl. Soft Comput..

[235]  Shengxiang Yang,et al.  Bi-goal evolution for many-objective optimization problems , 2015, Artif. Intell..

[236]  Ivan Petrovic,et al.  Modification of the Best-Worst and MABAC methods: A novel approach based on interval-valued fuzzy-rough numbers , 2018, Expert Syst. Appl..

[237]  Markus Wagner,et al.  Evolutionary many-objective optimization: A quick-start guide , 2015 .

[238]  Schalk Kok,et al.  MOTA: A Many-Objective Tuning Algorithm Specialized for Tuning under Multiple Objective Function Evaluation Budgets , 2017, Evolutionary Computation.

[239]  B. B. Zaidan,et al.  Novel Methodology for Triage and Prioritizing Using "Big Data" Patients with Chronic Heart Diseases Through Telemedicine Environmental , 2017, Int. J. Inf. Technol. Decis. Mak..

[240]  Yanli Yin,et al.  Energy-Efficient Many-Objective Virtual Machine Placement Optimization in a Cloud Computing Environment , 2017, IEEE Access.

[241]  Ye Tian,et al.  A Decision Variable Clustering-Based Evolutionary Algorithm for Large-Scale Many-Objective Optimization , 2018, IEEE Transactions on Evolutionary Computation.

[242]  Rammohan Mallipeddi,et al.  Pareto Dominance-Based Algorithms With Ranking Methods for Many-Objective Optimization , 2017, IEEE Access.

[243]  Shahram Sarkani,et al.  Many-objective stochastic path finding using reinforcement learning , 2017, Expert Syst. Appl..

[244]  B. B. Zaidan,et al.  Fault-Tolerant mHealth Framework in the Context of IoT-Based Real-Time Wearable Health Data Sensors , 2019, IEEE Access.

[245]  Heike Trautmann,et al.  Multiobjective evolutionary algorithms based on target region preferences , 2018, Swarm Evol. Comput..

[246]  Hiroyuki Sato,et al.  Chain-reaction solution update in MOEA/D and its effects on multi- and many-objective optimization , 2016, Soft Comput..

[247]  Aurora Trinidad Ramirez Pozo,et al.  Parallel multi-swarm PSO strategies for solving many objective optimization problems , 2019, J. Parallel Distributed Comput..

[248]  Sanghamitra Bandyopadhyay,et al.  DECOR: Differential Evolution using Clustering based Objective Reduction for many-objective optimization , 2018, Inf. Sci..

[249]  Qingfu Zhang,et al.  Comparison between MOEA/D and NSGA-III on a set of novel many and multi-objective benchmark problems with challenging difficulties , 2019, Swarm Evol. Comput..

[250]  Jafar Rezaei,et al.  Measuring the relative importance of the logistics performance index indicators using Best Worst Method , 2018, Transport Policy.

[251]  B. B. Zaidan,et al.  Survey on fuzzy TOPSIS state-of-the-art between 2007 and 2017 , 2019, Comput. Oper. Res..

[252]  Jinhua Zheng,et al.  A novel metric based on changes in pareto domination ratio for objective reduction of many-objective optimization problems , 2017, J. Exp. Theor. Artif. Intell..

[253]  João A. Vasconcelos,et al.  LONSA: A labeling-oriented non-dominated sorting algorithm for evolutionary many-objective optimization , 2018, Swarm Evol. Comput..

[254]  Gwo-Hshiung Tzeng,et al.  A VIKOR technique based on DEMATEL and ANP for information security risk control assessment , 2013, Inf. Sci..

[255]  Xifan Yao,et al.  A decomposition and statistical learning based many-objective artificial bee colony optimizer , 2019, Inf. Sci..

[256]  Eva-Maria Nordström,et al.  Decision support for participatory forest planning using AHP and TOPSIS. , 2016 .

[257]  Qing Yang,et al.  Evaluation and Classification of Overseas Talents in China Based on the BWM for Intuitionistic Relations , 2016, Symmetry.

[258]  Jafar Y. Al-Jawad,et al.  A comprehensive optimum integrated water resources management approach for multidisciplinary water resources management problems. , 2019, Journal of environmental management.

[259]  Yaochu Jin,et al.  A Many-Objective Evolutionary Algorithm Using A One-by-One Selection Strategy , 2017, IEEE Transactions on Cybernetics.

[260]  Mohd Bakri Ishak,et al.  Optimal selection of Iron and Steel wastewater treatment technology using integrated multi-criteria decision-making techniques and fuzzy logic , 2017 .

[261]  Chao Wang,et al.  A niche-elimination operation based NSGA-III algorithm for many-objective optimization , 2017, Applied Intelligence.

[262]  Yiu-ming Cheung,et al.  Objective Extraction for Many-Objective Optimization Problems: Algorithm and Test Problems , 2016, IEEE Transactions on Evolutionary Computation.

[263]  David A. Lowther,et al.  Visualization and Analysis of Tradeoffs in Many-Objective Optimization: A Case Study on the Interior Permanent Magnet Motor Design , 2016, IEEE Transactions on Magnetics.

[264]  Ye Tian,et al.  A region division based diversity maintaining approach for many-objective optimization , 2017, Integr. Comput. Aided Eng..

[265]  Pui-In Mak,et al.  Many-Objective Sizing Optimization of a Class-C/D VCO for Ultralow-Power IoT and Ultralow-Phase-Noise Cellular Applications , 2019, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[266]  A. A. Zaidan,et al.  Novel technique for reorganisation of opinion order to interval levels for solving several instances representing prioritisation in patients with multiple chronic diseases , 2019, Comput. Methods Programs Biomed..

[267]  Adrian Iftene,et al.  Dynamic Objective Sampling in Many-objective Optimization , 2015, KES.