HMOBEDA: Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm

Probabilistic modeling of selected solutions and incorporation of local search methods are approaches that can notably improve the results of multi-objective evolutionary algorithms (MOEAs). In the past, these approaches have been jointly applied to multi-objective problems (MOPs) with excellent results. In this paper, we introduce for the first time a joint probabilistic modeling of (1) local search methods with (2) decision variables and (3) the objectives in a framework named HMOBEDA. The proposed approach is compared with six evolutionary methods (including a modified version of NSGA-III, adapted to solve combinatorial optimization) on instances of the multi-objective knapsack problem with 3, 4, and 5 objectives. Results show that HMOBEDA is a competitive approach. It outperforms the other methods according to the hypervolume indicator.

[1]  Qingfu Zhang,et al.  An Estimation of Distribution Algorithm With Cheap and Expensive Local Search Methods , 2015, IEEE Transactions on Evolutionary Computation.

[2]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[3]  A. Tamhane,et al.  Multiple Comparison Procedures , 1989 .

[4]  Marco Laumanns,et al.  Bayesian Optimization Algorithms for Multi-objective Optimization , 2002, PPSN.

[5]  Dipankar Dasgupta,et al.  An empirical comparison of memetic algorithm strategies on the multiobjective quadratic assignment problem , 2009, 2009 IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making(MCDM).

[6]  Richard E. Neapolitan,et al.  Learning Bayesian networks , 2007, KDD '07.

[7]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints , 2014, IEEE Transactions on Evolutionary Computation.

[8]  Daniel Zelterman,et al.  Bayesian Artificial Intelligence , 2005, Technometrics.

[9]  David J. Groggel,et al.  Practical Nonparametric Statistics , 2000, Technometrics.

[10]  Ye Xu,et al.  An effective hybrid EDA-based algorithm for solving multidimensional knapsack problem , 2012, Expert Syst. Appl..

[11]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[12]  Qingfu Zhang,et al.  An evolutionary algorithm with guided mutation for the maximum clique problem , 2005, IEEE Transactions on Evolutionary Computation.

[13]  Hyun-Tae Kim,et al.  A hybrid multiobjective evolutionary algorithm: Striking a balance with local search , 2010, Math. Comput. Model..

[14]  J. A. Lozano,et al.  Towards a New Evolutionary Computation: Advances on Estimation of Distribution Algorithms (Studies in Fuzziness and Soft Computing) , 2006 .

[15]  Judea Pearl,et al.  Chapter 2 – BAYESIAN INFERENCE , 1988 .

[16]  Marco Laumanns,et al.  PISA: A Platform and Programming Language Independent Interface for Search Algorithms , 2003, EMO.

[17]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[18]  Edmund K. Burke,et al.  Multimeme Algorithms for Protein Structure Prediction , 2002, PPSN.

[19]  Hisao Ishibuchi,et al.  Behavior of Multiobjective Evolutionary Algorithms on Many-Objective Knapsack Problems , 2015, IEEE Transactions on Evolutionary Computation.

[20]  Qingfu Zhang,et al.  Hybrid Estimation of Distribution Algorithm for Multiobjective Knapsack Problem , 2004, EvoCOP.

[21]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[22]  Concha Bielza,et al.  Multiobjective Estimation of Distribution Algorithm Based on Joint Modeling of Objectives and Variables , 2014, IEEE Transactions on Evolutionary Computation.

[23]  El-Ghazali Talbi,et al.  Metaheuristics - From Design to Implementation , 2009 .

[24]  RåHW Fkryd.  Multiobjective Bayesian Optimization Algorithm for Combinatorial Problems : Theory and practice , 2002 .

[25]  Hisao Ishibuchi,et al.  Preference-based NSGA-II for many-objective knapsack problems , 2014, 2014 Joint 7th International Conference on Soft Computing and Intelligent Systems (SCIS) and 15th International Symposium on Advanced Intelligent Systems (ISIS).

[26]  Natalio Krasnogor,et al.  Toward truly "memetic" memetic algorithms: discussion and proof of concepts , 2002 .

[27]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point Based Nondominated Sorting Approach, Part II: Handling Constraints and Extending to an Adaptive Approach , 2014, IEEE Transactions on Evolutionary Computation.

[28]  Constantin F. Aliferis,et al.  The max-min hill-climbing Bayesian network structure learning algorithm , 2006, Machine Learning.

[29]  Kevin B. Korb,et al.  Bayesian Artificial Intelligence, Second Edition , 2010 .

[30]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[31]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[32]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[33]  Hisao Ishibuchi,et al.  Implementation of Multiobjective Memetic Algorithms for Combinatorial Optimization Problems: A Knapsack Problem Case Study , 2009 .

[34]  Kalyanmoy Deb,et al.  Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms , 1994, Evolutionary Computation.

[35]  Hisao Ishibuchi,et al.  Scalability of multiobjective genetic local search to many-objective problems: Knapsack problem case studies , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[36]  H. Keselman,et al.  Multiple Comparison Procedures , 2005 .

[37]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[38]  Alcione de Paiva Oliveira,et al.  Multi-objective Variable Neighborhood Search Algorithms for a Single Machine Scheduling Problem with Distinct due Windows , 2011, CLEI Selected Papers.

[39]  Concha Bielza,et al.  A review on probabilistic graphical models in evolutionary computation , 2012, Journal of Heuristics.

[41]  Robert L. Wolpert,et al.  Statistical Inference , 2019, Encyclopedia of Social Network Analysis and Mining.

[42]  Josef Schwarz,et al.  PARETO BAYESIAN OPTIMIZATION ALGORITHM FOR THE MULTIOBJECTIVE 0/1 KNAPSACK PROBLEM , 2002 .

[43]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[44]  S. Miyano,et al.  Finding Optimal Bayesian Network Given a Super-Structure , 2008 .

[45]  David E. Goldberg,et al.  Multi-objective bayesian optimization algorithm , 2002 .

[46]  Pedro Larrañaga,et al.  Combining variable neighborhood search and estimation of distribution algorithms in the protein side chain placement problem , 2007, J. Heuristics.

[47]  Changhe Yuan,et al.  Learning Optimal Bayesian Networks: A Shortest Path Perspective , 2013, J. Artif. Intell. Res..

[48]  John J. Bartholdi,et al.  The Knapsack Problem , 2008 .

[49]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[50]  Balbal Samir Balbal Samir,et al.  Local Search Heuristic for Multiple Knapsack Problem , 2012 .