Multi-Objective Optimization with an Adaptive Resonance Theory-Based Estimation of Distribution Algorithm: A Comparative Study

The introduction of learning to the search mechanisms of optimization algorithms has been nominated as one of the viable approaches when dealing with complex optimization problems, in particular with multi-objective ones. One of the forms of carrying out this hybridization process is by using multi-objective optimization estimation of distribution algorithms (MOEDAs). However, it has been pointed out that current MOEDAs have a intrinsic shortcoming in their model-building algorithms that hamper their performance. In this work we argue that error-based learning, the class of learning most commonly used in MOEDAs is responsible for current MOEDA underachievement. We present adaptive resonance theory (ART) as a suitable learning paradigm alternative and present a novel algorithm called multi-objective ART-based EDA (MARTEDA) that uses a Gaussian ART neural network for model-building and an hypervolume-based selector as described for the HypE algorithm. In order to assert the improvement obtained by combining two cutting-edge approaches to optimization an extensive set of experiments are carried out. These experiments also test the scalability of MARTEDA as the number of objective functions increases.

[1]  Beat Kleiner,et al.  Graphical Methods for Data Analysis , 1983 .

[2]  David B. Fogel,et al.  The Burden of Proof: Part II , 2008, WCCI.

[3]  Karl-Heinz Waldmann,et al.  Operations Research Proceedings 2006 , 2007 .

[4]  H. B. Mann,et al.  On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other , 1947 .

[5]  Carlos M. Fonseca,et al.  An Improved Dimension-Sweep Algorithm for the Hypervolume Indicator , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[6]  David W. Corne,et al.  Learning-assisted evolutionary search for scalable function optimization: LEM(ID3) , 2010, IEEE Congress on Evolutionary Computation.

[7]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[8]  Witold Kosiński,et al.  Advances in Evolutionary Algorithms , 2008 .

[9]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[10]  R. Lyndon While,et al.  A faster algorithm for calculating hypervolume , 2006, IEEE Transactions on Evolutionary Computation.

[11]  Lothar Thiele,et al.  A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers , 2006 .

[12]  J. A. Lozano,et al.  Towards a New Evolutionary Computation: Advances on Estimation of Distribution Algorithms (Studies in Fuzziness and Soft Computing) , 2006 .

[13]  Peter J. Fleming,et al.  On the Evolutionary Optimization of Many Conflicting Objectives , 2007, IEEE Transactions on Evolutionary Computation.

[14]  Jonathan L. Shapiro,et al.  Diversity Loss in General Estimation of Distribution Algorithms , 2006, PPSN.

[15]  S. Grossberg Studies of mind and brain : neural principles of learning, perception, development, cognition, and motor control , 1982 .

[16]  Tobias Friedrich,et al.  Approximating the Volume of Unions and Intersections of High-Dimensional Geometric Objects , 2008, ISAAC.

[17]  David W. Corne,et al.  The simplest evolution/learning hybrid: LEM with KNN , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[18]  R. Lyndon While,et al.  A review of multiobjective test problems and a scalable test problem toolkit , 2006, IEEE Transactions on Evolutionary Computation.

[19]  Jesús García,et al.  Introducing MONEDA: scalable multiobjective optimization with a neural estimation of distribution algorithm , 2008, GECCO '08.

[20]  Eckart Zitzler,et al.  HypE: An Algorithm for Fast Hypervolume-Based Many-Objective Optimization , 2011, Evolutionary Computation.

[21]  Riccardo Poli,et al.  Genetic and Evolutionary Computation , 2006, Intelligenza Artificiale.

[22]  Kalyanmoy Deb,et al.  Faster Hypervolume-Based Search Using Monte Carlo Sampling , 2008, MCDM.

[23]  John M. Chambers,et al.  Graphical Methods for Data Analysis , 1983 .

[24]  Kalyanmoy Deb,et al.  On Handling a Large Number of Objectives A Posteriori and During Optimization , 2008, Multiobjective Problem Solving from Nature.

[25]  M. E. Muller,et al.  A Note on the Generation of Random Normal Deviates , 1958 .

[26]  Qingfu Zhang,et al.  An evolutionary algorithm with guided mutation for the maximum clique problem , 2005, IEEE Transactions on Evolutionary Computation.

[27]  Nicola Beume,et al.  S-Metric Calculation by Considering Dominated Hypervolume as Klee's Measure Problem , 2009, Evolutionary Computation.

[28]  Nicola Beume,et al.  SMS-EMOA: Multiobjective selection based on dominated hypervolume , 2007, Eur. J. Oper. Res..

[29]  Martin Pelikan,et al.  Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications (Studies in Computational Intelligence) , 2006 .

[30]  Moonis Ali,et al.  Innovations in Applied Artificial Intelligence , 2005 .

[31]  Ryszard S. Michalski,et al.  LEARNABLE EVOLUTION MODEL: Evolutionary Processes Guided by Machine Learning , 2004, Machine Learning.

[32]  Pedro Larrañaga,et al.  GA-EDA: hybrid evolutionary algorithm using genetic and estimation of distribution algorithms , 2004 .

[33]  Chang Wook Ahn,et al.  Multiobjective real-coded bayesian optimization algorithmrevisited: diversity preservation , 2007, GECCO '07.

[34]  Charles Gide,et al.  Cours d'économie politique , 1911 .

[35]  Stephen Grossberg,et al.  Studies of mind and brain , 1982 .

[36]  Carlos A. Coello Coello,et al.  GIAA TECHNINAL REPORT GIAA2010E001 On Current Model-Building Methods for Multi-Objective Estimation of Distribution Algorithms: Shortcommings and Directions for Improvement , 2010 .

[37]  Dirk Thierens,et al.  The Naive MIDEA: A Baseline Multi-objective EA , 2005, EMO.

[38]  Jyrki Wallenius,et al.  Bibliometric Analysis of Multiple Criteria Decision Making/Multiattribute Utility Theory , 2008, MCDM.

[39]  Kaisa Miettinen,et al.  Nonlinear multiobjective optimization , 1998, International series in operations research and management science.

[40]  Eckart Zitzler,et al.  Dimensionality Reduction in Multiobjective Optimization: The Minimum Objective Subset Problem , 2006, OR.

[41]  Theodor J. Stewart,et al.  Real-World Applications of Multiobjective Optimization , 2008, Multiobjective Optimization.

[42]  Selmer Bringsjord,et al.  P=np , 2004, ArXiv.

[43]  K. Deb,et al.  On Finding Pareto-Optimal Solutions Through Dimensionality Reduction for Certain Large-Dimensional Multi-Objective Optimization Problems , 2022 .

[44]  Chang Wook Ahn,et al.  Advances in Evolutionary Algorithms: Theory, Design and Practice , 2006, Studies in Computational Intelligence.

[45]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems (Genetic and Evolutionary Computation) , 2006 .

[46]  Johannes Bader,et al.  Hypervolume-based search for multiobjective optimization: Theory and methods , 2010 .

[47]  Nicola Beume,et al.  Pareto-, Aggregation-, and Indicator-Based Methods in Many-Objective Optimization , 2007, EMO.

[48]  James R. Williamson,et al.  Gaussian ARTMAP: A Neural Network for Fast Incremental Learning of Noisy Multidimensional Maps , 1996, Neural Networks.

[49]  Edmund K. Burke,et al.  Parallel Problem Solving from Nature - PPSN IX: 9th International Conference, Reykjavik, Iceland, September 9-13, 2006, Proceedings , 2006, PPSN.

[50]  Marcus Gallagher,et al.  On the importance of diversity maintenance in estimation of distribution algorithms , 2005, GECCO '05.

[51]  David E. Goldberg,et al.  Multiobjective Estimation of Distribution Algorithms , 2006, Scalable Optimization via Probabilistic Modeling.

[52]  Kim Fung Man,et al.  Multiobjective Optimization , 2011, IEEE Microwave Magazine.

[53]  Jesús García,et al.  Moving away from error-based learning in multi-objective estimation of distribution algorithms , 2010, GECCO '10.