A supervised learning approach involving active subspaces for an efficient genetic algorithm in high-dimensional optimization problems

In this work, we present an extension of the genetic algorithm (GA) which exploits the active subspaces (AS) property to evolve the individuals on a lower dimensional space. In many cases, GA requires in fact more function evaluations than others optimization method to converge to the optimum. Thus, complex and high-dimensional functions may result intractable with the standard algorithm. To address this issue, we propose to linearly map the input parameter space of the original function onto its AS before the evolution, performing the mutation and mate processes in a lower dimensional space. In this contribution, we describe the novel method called ASGA, presenting differences and similarities with the standard GA method. We test the proposed method over n-dimensional benchmark functions -- Rosenbrock, Ackley, Bohachevsky, Rastrigin, Schaffer N. 7, and Zakharov -- and finally we apply it to an aeronautical shape optimization problem.

[1]  G. Rozza,et al.  An integrated data-driven computational pipeline with model order reduction for industrial and applied mathematics , 2018, 1810.12364.

[2]  Jan S. Hesthaven,et al.  Reduced order modeling for nonlinear structural analysis using Gaussian process regression , 2018, Computer Methods in Applied Mechanics and Engineering.

[3]  Jiaxin Zhang,et al.  Learning nonlinear level sets for dimensionality reduction in function approximation , 2019, NeurIPS.

[4]  Patrick Siarry,et al.  A Continuous Genetic Algorithm Designed for the Global Optimization of Multimodal Functions , 2000, J. Heuristics.

[5]  Gianluigi Rozza,et al.  Hull shape design optimization with parameter space and model reductions, and self-learning mesh morphing , 2021, Journal of Marine Science and Engineering.

[6]  Ivan Oseledets,et al.  Active Subspace of Neural Networks: Structural Analysis and Universal Attacks , 2020, SIAM J. Math. Data Sci..

[7]  Yang Yu,et al.  Derivative-Free Optimization of High-Dimensional Non-Convex Functions by Sequential Random Embeddings , 2016, IJCAI.

[8]  Robert L. Smith The hit-and-run sampler: a globally reaching Markov chain sampler for generating arbitrary multivariate distributions , 1996, Winter Simulation Conference.

[9]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[10]  R. M. Hicks,et al.  Wing Design by Numerical Optimization , 1977 .

[11]  Ling Li,et al.  Sequential design of computer experiments for the estimation of a probability of failure , 2010, Statistics and Computing.

[12]  Ruhul A. Sarker,et al.  A new genetic algorithm for solving optimization problems , 2014, Eng. Appl. Artif. Intell..

[13]  Zoubin Ghahramani,et al.  Unifying linear dimensionality reduction , 2014, 1406.0873.

[14]  Martin D. Buhmann,et al.  Radial Basis Functions: Theory and Implementations: Preface , 2003 .

[15]  Paul G. Constantine,et al.  Active Subspaces - Emerging Ideas for Dimension Reduction in Parameter Studies , 2015, SIAM spotlights.

[16]  Robert L. Smith,et al.  Hit-and-Run Algorithms for Generating Multivariate Distributions , 1993, Math. Oper. Res..

[17]  I. H. Abbott,et al.  Theory of Wing Sections: Including a Summary of Airfoil Data , 1959 .

[18]  Xin-She Yang,et al.  A literature survey of benchmark functions for global optimisation problems , 2013, Int. J. Math. Model. Numer. Optimisation.

[19]  D. Ginsbourger,et al.  A benchmark of kriging-based infill criteria for noisy optimization , 2013, Structural and Multidisciplinary Optimization.

[20]  Gianluigi Rozza,et al.  Dimension reduction in heterogeneous parametric spaces with application to naval engineering shape design problems , 2017, Advanced Modeling and Simulation in Engineering Sciences.

[21]  Coralia Cartis,et al.  A dimensionality reduction technique for unconstrained global optimization of functions with low effective dimensionality , 2020, Information and Inference: A Journal of the IMA.

[22]  Ernesto P. Adorio,et al.  MVF - Multivariate Test Functions Library in C for Unconstrained Global Optimization , 2005 .

[23]  Gianluigi Rozza,et al.  A non-intrusive approach for the reconstruction of POD modal coefficients through active subspaces , 2019 .

[24]  Martin W. Hess,et al.  Basic Ideas and Tools for Projection-Based Model Reduction of Parametric Partial Differential Equations , 2019, Snapshot-Based Methods and Algorithms.

[25]  Claus Emmeche,et al.  The garden in the machine: the emerging science of artificial life , 1994 .

[26]  Rafael Martí,et al.  Experimental Testing of Advanced Scatter Search Designs for Global Optimization of Multimodal Functions , 2005, J. Glob. Optim..

[27]  Youssef Marzouk,et al.  Gradient-Based Dimension Reduction of Multivariate Vector-Valued Functions , 2020, SIAM J. Sci. Comput..

[28]  Qiqi Wang,et al.  Erratum: Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces , 2013, SIAM J. Sci. Comput..

[29]  Rajarshi Das,et al.  A Study of Control Parameters Affecting Online Performance of Genetic Algorithms for Function Optimization , 1989, ICGA.

[30]  Ata Kabán,et al.  REMEDA: Random Embedding EDA for Optimising Functions with Intrinsic Dimension , 2016, PPSN.

[31]  Yurii Nesterov,et al.  Random Gradient-Free Minimization of Convex Functions , 2015, Foundations of Computational Mathematics.

[32]  Manoj Kumar,et al.  Genetic Algorithm: Review and Application , 2010 .

[33]  John Holland,et al.  Adaptation in Natural and Artificial Sys-tems: An Introductory Analysis with Applications to Biology , 1975 .

[34]  Juan J. Alonso,et al.  Active Subspaces for Shape Optimization , 2014 .

[35]  Krzysztof Choromanski,et al.  From Complexity to Simplicity: Adaptive ES-Active Subspaces for Blackbox Optimization , 2019, NeurIPS.

[36]  Michael B. Wakin,et al.  Computing active subspaces efficiently with gradient sketching , 2015, 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[37]  Nando de Freitas,et al.  Bayesian Optimization in a Billion Dimensions via Random Embeddings , 2013, J. Artif. Intell. Res..

[38]  Robert A. Bridges,et al.  Active Manifolds: A non-linear analogue to Active Subspaces , 2019, ICML.

[39]  R. Hinterding,et al.  Gaussian mutation and self-adaption for numeric genetic algorithms , 1995, Proceedings of 1995 IEEE International Conference on Evolutionary Computation.

[40]  Heinz Mühlenbein,et al.  The parallel genetic algorithm as function optimizer , 1991, Parallel Comput..

[41]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[42]  Gianluigi Rozza,et al.  Shape optimization through proper orthogonal decomposition with interpolation and dynamic mode decomposition enhanced by active subspaces , 2019, 1905.05483.

[43]  Gianluigi Rozza,et al.  ATHENA: Advanced Techniques for High Dimensional Parameter Spaces to Enhance Numerical Analysis , 2021, Softw. Impacts.

[44]  G. Rozza,et al.  Combined Parameter and Model Reduction of Cardiovascular Problems by Means of Active Subspaces and POD-Galerkin Methods , 2017, 1711.10884.

[45]  Zvi Drezner,et al.  A New Genetic Algorithm for the Quadratic Assignment Problem , 2003, INFORMS J. Comput..

[46]  John H. Holland,et al.  Genetic Algorithms and the Optimal Allocation of Trials , 1973, SIAM J. Comput..

[47]  V. L. Turova-Botkina,et al.  An algorithm for finding the Chebyshev center of a convex polyhedron , 1994, System Modelling and Optimization.

[48]  S. Vempala,et al.  Hit-and-Run from a Corner , 2006 .

[49]  Gianluigi Rozza,et al.  Model Order Reduction by means of Active Subspaces and Dynamic Mode Decomposition for Parametric Hull Shape Design Hydrodynamics , 2018, 1803.07377.

[50]  Zelda B. Zabinsky,et al.  A Numerical Evaluation of Several Stochastic Algorithms on Selected Continuous Global Optimization Test Problems , 2005, J. Glob. Optim..

[51]  R.SIVARAJ,et al.  A REVIEW OF SELECTION METHODS IN GENETIC ALGORITHM , 2011 .

[52]  Gianluigi Rozza,et al.  Kernel‐based active subspaces with application to computational fluid dynamics parametric problems using the discontinuous Galerkin method , 2020, International journal for numerical methods in engineering.