A Multipopulation Evolutionary Algorithm for Solving Large-Scale Multimodal Multiobjective Optimization Problems

Multimodal multiobjective optimization problems (MMOPs) widely exist in real-world applications, which have multiple equivalent Pareto-optimal solutions that are similar in the objective space but totally different in the decision space. While some evolutionary algorithms (EAs) have been developed to find the equivalent Pareto-optimal solutions in recent years, they are ineffective to handle large-scale MMOPs having a large number of variables. This article thus proposes an EA for solving large-scale MMOPs with sparse Pareto-optimal solutions, i.e., most variables in the optimal solutions are 0. The proposed algorithm explores different regions of the decision space via multiple subpopulations and guides the search behavior of the subpopulations via adaptively updated guiding vectors. The guiding vector for each subpopulation not only provides efficient convergence in the huge search space but also differentiates its search direction from others to handle the multimodality. While most existing EAs solve MMOPs with 2–7 decision variables, the proposed algorithm is shown to be effective for benchmark MMOPs with up to 500 decision variables. Moreover, the proposed algorithm also produces a better result than state-of-the-art methods for the neural architecture search.

[1]  Jun Zhang,et al.  Distributed Differential Evolution Based on Adaptive Mergence and Split for Large-Scale Optimization , 2018, IEEE Transactions on Cybernetics.

[2]  Ramesh Raskar,et al.  Designing Neural Network Architectures using Reinforcement Learning , 2016, ICLR.

[3]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[4]  Ofer M. Shir,et al.  Enhancing Decision Space Diversity in Evolutionary Multiobjective Algorithms , 2009, EMO.

[5]  Ye Tian,et al.  PlatEMO: A MATLAB Platform for Evolutionary Multi-Objective Optimization [Educational Forum] , 2017, IEEE Computational Intelligence Magazine.

[6]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[7]  Fang Liu,et al.  A Multiobjective Evolutionary Algorithm Based on Decision Variable Analyses for Multiobjective Optimization Problems With Large-Scale Variables , 2016, IEEE Transactions on Evolutionary Computation.

[8]  Xiaodong Li,et al.  Seeking Multiple Solutions: An Updated Survey on Niching Methods and Their Applications , 2017, IEEE Transactions on Evolutionary Computation.

[9]  Hisao Ishibuchi,et al.  A Framework to Handle Multimodal Multiobjective Optimization in Decomposition-Based Evolutionary Algorithms , 2020, IEEE Transactions on Evolutionary Computation.

[10]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[11]  Hisao Ishibuchi,et al.  A Review of Evolutionary Multi-modal Multi-objective Optimization , 2020, arXiv.org.

[12]  Ye Tian,et al.  A Decision Variable Clustering-Based Evolutionary Algorithm for Large-Scale Many-Objective Optimization , 2018, IEEE Transactions on Evolutionary Computation.

[13]  Kalyanmoy Deb,et al.  Simulated Binary Crossover for Continuous Search Space , 1995, Complex Syst..

[14]  Alok Aggarwal,et al.  Regularized Evolution for Image Classifier Architecture Search , 2018, AAAI.

[15]  Oliver Kramer,et al.  DBSCAN-based multi-objective niching to approximate equivalent pareto-subsets , 2010, GECCO '10.

[16]  Zexuan Zhu,et al.  Multimodal Multiobjective Evolutionary Optimization With Dual Clustering in Decision and Objective Spaces , 2021, IEEE Transactions on Evolutionary Computation.

[17]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Qingfu Zhang,et al.  Approximating the Set of Pareto-Optimal Solutions in Both the Decision and Objective Spaces by an Estimation of Distribution Algorithm , 2009, IEEE Transactions on Evolutionary Computation.

[19]  Hisao Ishibuchi,et al.  A Decomposition-based Large-scale Multi-modal Multi-objective optimization Algorithm , 2020, 2020 IEEE Congress on Evolutionary Computation (CEC).

[20]  Ye Tian,et al.  An Evolutionary Algorithm for Large-Scale Sparse Multiobjective Optimization Problems , 2020, IEEE Transactions on Evolutionary Computation.

[21]  Kalyanmoy Deb,et al.  Omni-optimizer: A Procedure for Single and Multi-objective Optimization , 2005, EMO.

[22]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[23]  Jing J. Liang,et al.  Multimodal multi-objective optimization: A preliminary study , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[24]  Kalyanmoy Deb,et al.  Improved Pruning of Non-Dominated Solutions Based on Crowding Distance for Bi-Objective Optimization Problems , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[25]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[26]  Wei Wu,et al.  Practical Block-Wise Neural Network Architecture Generation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[27]  Charu C. Aggarwal,et al.  On the Surprising Behavior of Distance Metrics in High Dimensional Spaces , 2001, ICDT.

[28]  Hisao Ishibuchi,et al.  A Scalable Multimodal Multiobjective Test Problem , 2019, 2019 IEEE Congress on Evolutionary Computation (CEC).

[29]  Jing J. Liang,et al.  A Multiobjective Particle Swarm Optimizer Using Ring Topology for Solving Multimodal Multiobjective Problems , 2018, IEEE Transactions on Evolutionary Computation.

[30]  Hisao Ishibuchi,et al.  A Decomposition-Based Evolutionary Algorithm for Multi-modal Multi-objective Optimization , 2018, PPSN.

[31]  Ye Tian,et al.  Efficient Large-Scale Multiobjective Optimization Based on a Competitive Swarm Optimizer , 2020, IEEE Transactions on Cybernetics.

[32]  Jing J. Liang,et al.  Multimodal Multiobjective Optimization in Feature Selection , 2019, 2019 IEEE Congress on Evolutionary Computation (CEC).

[33]  Theodore Lim,et al.  SMASH: One-Shot Model Architecture Search through HyperNetworks , 2017, ICLR.

[34]  Tomoyuki Hiroyasu,et al.  SPEA2+: Improving the Performance of the Strength Pareto Evolutionary Algorithm 2 , 2004, PPSN.

[35]  Hisao Ishibuchi,et al.  A Framework for Large-Scale Multiobjective Optimization Based on Problem Transformation , 2018, IEEE Transactions on Evolutionary Computation.

[36]  Shengxiang Yang,et al.  Shift-Based Density Estimation for Pareto-Based Algorithms in Many-Objective Optimization , 2014, IEEE Transactions on Evolutionary Computation.

[37]  Kalyanmoy Deb,et al.  NSGA-Net: neural architecture search using multi-objective genetic algorithm , 2018, GECCO.

[38]  Hisao Ishibuchi,et al.  Incorporation of a decision space diversity maintenance mechanism into MOEA/D for multi-modal multi-objective optimization , 2018, GECCO.

[39]  Quoc V. Le,et al.  Neural Architecture Search with Reinforcement Learning , 2016, ICLR.

[40]  Mark J. van der Laan,et al.  The relative performance of ensemble methods with deep convolutional neural networks for image classification , 2017, Journal of applied statistics.

[41]  Hisao Ishibuchi,et al.  Handling Imbalance Between Convergence and Diversity in the Decision Space in Evolutionary Multimodal Multiobjective Optimization , 2020, IEEE Transactions on Evolutionary Computation.

[42]  Mengjie Zhang,et al.  A New Two-Stage Evolutionary Algorithm for Many-Objective Optimization , 2019, IEEE Transactions on Evolutionary Computation.

[43]  Yang Yu,et al.  Solving High-Dimensional Multi-Objective Optimization Problems with Low Effective Dimensions , 2017, AAAI.

[44]  Hisao Ishibuchi,et al.  A Double-Niched Evolutionary Algorithm and Its Behavior on Polygon-Based Problems , 2018, PPSN.

[45]  Bernhard Sendhoff,et al.  Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[46]  Xiaodong Li,et al.  Niching Without Niching Parameters: Particle Swarm Optimization Using a Ring Topology , 2010, IEEE Transactions on Evolutionary Computation.

[47]  Ye Tian,et al.  An Efficient Approach to Nondominated Sorting for Evolutionary Multiobjective Optimization , 2015, IEEE Transactions on Evolutionary Computation.

[48]  Tomohiro Yoshikawa,et al.  A study on analysis of design variables in Pareto solutions for conceptual design optimization problem of hybrid rocket engine , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[49]  Jun Zhang,et al.  Learning Multimodal Parameters: A Bare-Bones Niching Differential Evolution Approach , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[50]  Tie-Yan Liu,et al.  Neural Architecture Optimization , 2018, NeurIPS.

[51]  Marcus Gallagher,et al.  Sampling Techniques and Distance Metrics in High Dimensional Continuous Landscape Analysis: Limitations and Improvements , 2014, IEEE Transactions on Evolutionary Computation.

[52]  M. Ehsan Shafiee,et al.  An evolutionary algorithm approach to generate distinct sets of non-dominated solutions for wicked problems , 2013, Eng. Appl. Artif. Intell..

[53]  Jing J. Liang,et al.  A cluster based PSO with leader updating mechanism and ring-topology for multimodal multi-objective optimization , 2019, Swarm Evol. Comput..

[54]  Luís A. Alexandre,et al.  Weighted Convolutional Neural Network Ensemble , 2014, CIARP.

[55]  Vijay Vasudevan,et al.  Learning Transferable Architectures for Scalable Image Recognition , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[56]  Jeremy J. Michalek,et al.  Architectural layout design optimization , 2002 .

[57]  Kalyanmoy Deb,et al.  A combined genetic adaptive search (GeneAS) for engineering design , 1996 .

[58]  Nikos Komodakis,et al.  Wide Residual Networks , 2016, BMVC.

[59]  Xin Yao,et al.  Accelerating Large-Scale Multiobjective Optimization via Problem Reformulation , 2019, IEEE Transactions on Evolutionary Computation.

[60]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[61]  Yiming Yang,et al.  DARTS: Differentiable Architecture Search , 2018, ICLR.

[62]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[63]  Quoc V. Le,et al.  Large-Scale Evolution of Image Classifiers , 2017, ICML.

[64]  Oriol Vinyals,et al.  Hierarchical Representations for Efficient Architecture Search , 2017, ICLR.

[65]  Gary G. Yen,et al.  A Multimodal Multiobjective Evolutionary Algorithm Using Two-Archive and Recombination Strategies , 2019, IEEE Transactions on Evolutionary Computation.