Exploring the probabilistic graphic model of a hybrid multi-objective Bayesian estimation of distribution algorithm

Abstract The Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm (HMOBEDA) has shown to be very competitive for Many Objective Optimization Problems (MaOPs). The Probabilistic Graphic Model (PGM) of HMOBEDA expands the possibilities for exploration as it provides the joint probability of decision variables, objectives, and configuration parameters of an embedded local search. This work investigates different sampling mechanisms of HMOBEDA, applying the considered approaches to two different combinatorial MaOPs. Moreover, the paper provides a broad set of statistical analyses on its PGM model. These analyses have been carried out to evaluate how the interactions among variables, objectives and local search parameters are captured by the model and how information collected from different runs can be aggregated and explored in a Probabilistic Pareto Front. In experiments, two variants of HMOBEDA are compared with the original version, each one with a different set of evidences fixed during the sampling process. Results for instances of multi-objective knapsack problem with 2–5 and 8 objectives show that the best variant outperforms the original HMOBEDA in terms of convergence and diversity in the solution set. This best variant is then compared with five state-of-the-art evolutionary algorithms using the knapsack problem instances as well as a set of MNK-landscape instances with 2, 3, 5 and 8 objectives. HMOBEDA outperforms all of them.

[1]  Hisao Ishibuchi,et al.  Preference-based NSGA-II for many-objective knapsack problems , 2014, 2014 Joint 7th International Conference on Soft Computing and Intelligent Systems (SCIS) and 15th International Symposium on Advanced Intelligent Systems (ISIS).

[3]  S. Kauffman,et al.  Towards a general theory of adaptive walks on rugged landscapes. , 1987, Journal of theoretical biology.

[4]  E. D. Weinberger,et al.  The NK model of rugged fitness landscapes and its application to maturation of the immune response. , 1989, Journal of theoretical biology.

[5]  Afif Masmoudi,et al.  A New Approach for Bayesian Classifier Learning Structure via K2 Algorithm , 2012, ICIC.

[6]  Carlos A. Coello Coello,et al.  HCS: A New Local Search Strategy for Memetic Multiobjective Evolutionary Algorithms , 2010, IEEE Transactions on Evolutionary Computation.

[7]  Kalyanmoy Deb,et al.  Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms , 1994, Evolutionary Computation.

[8]  Carolina P. de Almeida,et al.  Hybrid multi-objective Bayesian estimation of distribution algorithm: a comparative analysis for the multi-objective knapsack problem , 2017, Journal of Heuristics.

[9]  Linda C. van der Gaag,et al.  Inference and Learning in Multi-dimensional Bayesian Network Classifiers , 2007, ECSQARU.

[10]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[11]  Hisao Ishibuchi,et al.  Scalability of multiobjective genetic local search to many-objective problems: Knapsack problem case studies , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[12]  Constantin F. Aliferis,et al.  The max-min hill-climbing Bayesian network structure learning algorithm , 2006, Machine Learning.

[13]  P. Spirtes,et al.  Causation, prediction, and search , 1993 .

[14]  Constantin F. Aliferis,et al.  Algorithms for Large Scale Markov Blanket Discovery , 2003, FLAIRS.

[15]  Roberto Santana,et al.  Evolving MNK-landscapes with structural constraints , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[16]  Qingfu Zhang,et al.  An Estimation of Distribution Algorithm With Cheap and Expensive Local Search Methods , 2015, IEEE Transactions on Evolutionary Computation.

[17]  Dalessandro Soares Vianna,et al.  Local search-based heuristics for the multiobjective multidimensional knapsack problem , 2012 .

[18]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[19]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints , 2014, IEEE Transactions on Evolutionary Computation.

[20]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[21]  R. K. Ursem Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[22]  Kiyoshi Tanaka,et al.  Insights on properties of multiobjective MNK-landscapes , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[23]  David J. Groggel,et al.  Practical Nonparametric Statistics , 2000, Technometrics.

[24]  Sébastien Vérel,et al.  Global vs Local Search on Multi-objective NK-Landscapes: Contrasting the Impact of Problem Features , 2015, GECCO.

[25]  Stuart A. Kauffman,et al.  The origins of order , 1993 .

[26]  Johannes Bader,et al.  Hypervolume-based search for multiobjective optimization: Theory and methods , 2010 .

[27]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[28]  Josef Schwarz,et al.  PARETO BAYESIAN OPTIMIZATION ALGORITHM FOR THE MULTIOBJECTIVE 0/1 KNAPSACK PROBLEM , 2002 .

[29]  Larry Jenkins A bicriteria knapsack program for planning remediation of contaminated lightstation sites , 2002, Eur. J. Oper. Res..

[30]  Hisao Ishibuchi,et al.  Behavior of Multiobjective Evolutionary Algorithms on Many-Objective Knapsack Problems , 2015, IEEE Transactions on Evolutionary Computation.

[31]  Sébastien Vérel,et al.  A Feature-Based Performance Analysis in Evolutionary Multiobjective Optimization , 2015, EMO.

[32]  Qingfu Zhang,et al.  Hybrid Estimation of Distribution Algorithm for Multiobjective Knapsack Problem , 2004, EvoCOP.

[33]  Constantin F. Aliferis,et al.  Local Causal and Markov Blanket Induction for Causal Discovery and Feature Selection for Classification Part I: Algorithms and Empirical Evaluation , 2010, J. Mach. Learn. Res..

[34]  Jacques Teghem,et al.  The multiobjective multidimensional knapsack problem: a survey and a new approach , 2010, Int. Trans. Oper. Res..

[35]  Arnaud Fréville,et al.  Tabu Search Based Procedure for Solving the 0-1 MultiObjective Knapsack Problem: The Two Objectives Case , 2000, J. Heuristics.

[36]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[37]  Carolina P. de Almeida,et al.  Probabilistic Analysis of Pareto Front Approximation for a Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm , 2017, 2017 Brazilian Conference on Intelligent Systems (BRACIS).

[38]  Mariano Luque,et al.  Modified interactive chebyshev algorithm (MICA) for non-convex multiobjective programming , 2015, Optim. Lett..

[39]  Qingfu Zhang,et al.  Multiobjective evolutionary algorithms: A survey of the state of the art , 2011, Swarm Evol. Comput..

[40]  Martin V. Butz,et al.  Performance of evolutionary algorithms on NK landscapes with nearest neighbor interactions and tunable overlap , 2009, GECCO '09.

[41]  José M. Matías,et al.  Use of Bayesian networks classifiers for long-term mean wind turbine energy output estimation at a potential wind energy conversion site , 2011 .

[42]  Endika Bengoetxea,et al.  Inexact Graph Matching Using Estimation of Distribution Algorithms , 2002 .

[43]  Chuan-Kang Ting,et al.  Effect of model complexity for estimation of distribution algorithm in NK landscapes , 2013, 2013 IEEE Symposium on Foundations of Computational Intelligence (FOCI).

[44]  Linda C. van der Gaag,et al.  Multi-dimensional Bayesian Network Classifiers , 2006, Probabilistic Graphical Models.

[45]  Hisao Ishibuchi,et al.  Evolutionary many-objective optimization: A short review , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[46]  Marco Laumanns,et al.  Bayesian Optimization Algorithms for Multi-objective Optimization , 2002, PPSN.

[47]  Gwo-Hshiung Tzeng,et al.  A MULTIOBJECTIVE PROGRAMMING APPROACH FOR SELECTING NON-INDEPENDENT TRANSPORTATION INVESTMENT ALTERNATIVES , 1996 .

[48]  Ye Xu,et al.  An effective hybrid EDA-based algorithm for solving multidimensional knapsack problem , 2012, Expert Syst. Appl..

[49]  Yulan He,et al.  Choosing the best Bayesian classifier: an empirical study , 2009 .

[50]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[51]  M. Valente An NK-like model for complexity , 2014 .

[52]  Carolina P. de Almeida,et al.  HMOBEDA: Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm , 2016, GECCO.

[53]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[54]  Meir J. Rosenblatt,et al.  Generating the Discrete Efficient Frontier to the Capital Budgeting Problem , 1989, Oper. Res..

[55]  Daniel Zelterman,et al.  Bayesian Artificial Intelligence , 2005, Technometrics.

[56]  Sébastien Vérel,et al.  Pareto Local Optima of Multiobjective NK-Landscapes with Correlated Objectives , 2011, EvoCOP.

[57]  Jie Zhang,et al.  Consistencies and Contradictions of Performance Metrics in Multiobjective Optimization , 2014, IEEE Transactions on Cybernetics.

[58]  Marco Laumanns,et al.  PISA: A Platform and Programming Language Independent Interface for Search Algorithms , 2003, EMO.

[59]  Yong-Chang Jiao,et al.  MOEA/D with Uniform Design for Solving Multiobjective Knapsack Problems , 2013, J. Comput..

[60]  Andrzej Jaszkiewicz Comparison of local search-based metaheuristics on the multiple objective knapsack problem , 2001 .

[61]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[62]  Jin-Kao Hao,et al.  A Practical Case of the Multiobjective Knapsack Problem: Design, Modelling, Tests and Analysis , 2015, LION.

[63]  Qguhm -DVNLHZLF,et al.  On the performance of multiple objective genetic local search on the 0 / 1 knapsack problem . A comparative experiment , 2000 .

[64]  Aimin Zhou,et al.  A Multiobjective Evolutionary Algorithm Based on Decomposition and Preselection , 2015, BIC-TA.

[65]  Concha Bielza,et al.  A review on probabilistic graphical models in evolutionary computation , 2012, Journal of Heuristics.

[67]  Robert L. Wolpert,et al.  Statistical Inference , 2019, Encyclopedia of Social Network Analysis and Mining.

[68]  Concha Bielza,et al.  Multiobjective Estimation of Distribution Algorithm Based on Joint Modeling of Objectives and Variables , 2014, IEEE Transactions on Evolutionary Computation.

[69]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[70]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[71]  Gary B. Lamont,et al.  Multiobjective evolutionary algorithm test suites , 1999, SAC '99.

[72]  Kiyoshi Tanaka,et al.  Working principles, behavior, and performance of MOEAs on MNK-landscapes , 2007, Eur. J. Oper. Res..

[73]  Roberto Santana,et al.  Multi-objective NM-Landscapes , 2015, GECCO.

[74]  Michael M. Kostreva,et al.  Relocation problems arising in conservation biology , 1999 .

[75]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[76]  Martin Pelikan Analysis of estimation of distribution algorithms and genetic algorithms on NK landscapes , 2008, GECCO '08.

[77]  Changhe Yuan,et al.  Learning Optimal Bayesian Networks: A Shortest Path Perspective , 2013, J. Artif. Intell. Res..