A global optimization paradigm based on change of measures

A global optimization framework, COMBEO (Change Of Measure Based Evolutionary Optimization), is proposed. An important aspect in the development is a set of derivative-free additive directional terms, obtainable through a change of measures en route to the imposition of any stipulated conditions aimed at driving the realized design variables (particles) to the global optimum. The generalized setting offered by the new approach also enables several basic ideas, used with other global search methods such as the particle swarm or the differential evolution, to be rationally incorporated in the proposed set-up via a change of measures. The global search may be further aided by imparting to the directional update terms additional layers of random perturbations such as ‘scrambling’ and ‘selection’. Depending on the precise choice of the optimality conditions and the extent of random perturbation, the search can be readily rendered either greedy or more exploratory. As numerically demonstrated, the new proposal appears to provide for a more rational, more accurate and, in some cases, a faster alternative to many available evolutionary optimization schemes.

[1]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[2]  P. N. Suganthan,et al.  Differential Evolution: A Survey of the State-of-the-Art , 2011, IEEE Transactions on Evolutionary Computation.

[3]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[4]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[5]  D. Nualart.  The Malliavin Calculus and Related Topics , 1995 .

[6]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[7]  Louis A. Romero,et al.  Robust two-dimensional weighted and unweighted phase unwrapping that uses fast transforms and iterative methods , 1994 .

[8]  Mehmet Fatih Tasgetiren,et al.  Differential evolution algorithm with ensemble of parameters and mutation strategies , 2011, Appl. Soft Comput..

[9]  Jürgen Branke,et al.  Survey: State of the Art , 2002 .

[10]  M. Freidlin,et al.  Random Perturbations of Dynamical Systems , 1984 .

[11]  Anja Walter,et al.  Introduction To Stochastic Calculus With Applications , 2016 .

[12]  A. E. Eiben,et al.  On Evolutionary Exploration and Exploitation , 1998, Fundam. Informaticae.

[13]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[14]  S. Varadhan,et al.  On the Support of Diffusion Processes with Applications to the Strong Maximum Principle , 1972 .

[15]  Debasish Roy,et al.  Quantitative estimation of density variation in high-speed flows through inversion of the measured wavefront distortion , 2014 .

[16]  Holger H. Hoos,et al.  Automated Algorithm Configuration and Parameter Tuning , 2012, Autonomous Search.

[17]  Jean-Michel Renders,et al.  Hybrid methods using genetic algorithms for global optimization , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[18]  Debasish Roy,et al.  An Ensemble Kushner-Stratonovich (EnKS) Nonlinear Filter: Additive Particle Updates in Non-Iterative and Iterative Forms , 2014 .

[19]  S. Aachen.  Stochastic Differential Equations An Introduction With Applications , 2016 .

[20]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[21]  Jun Sun,et al.  Parameter estimation for chaotic systems with a Drift Particle Swarm Optimization method , 2010 .

[22]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[23]  P. Bickel,et al.  Obstacles to High-Dimensional Particle Filtering , 2008 .

[24]  Thomas Bartz-Beielstein,et al.  Experimental Methods for the Analysis of Optimization Algorithms , 2010 .

[26]  F. Glover,et al.  Handbook of Metaheuristics , 2019, International Series in Operations Research & Management Science.

[27]  R M Vasu,et al.  A Kushner–Stratonovich Monte Carlo filter applied to nonlinear dynamical system identification , 2014 .

[28]  Emile H. L. Aarts,et al.  Performance of the simulated annealing algorithm , 1987 .

[29]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[30]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[31]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[32]  Kenneth Sörensen,et al.  Metaheuristics - the metaphor exposed , 2015, Int. Trans. Oper. Res..

[33]  Kathryn A. Dowsland,et al.  Simulated Annealing , 1989, Encyclopedia of GIS.

[34]  Ville Tirronen,et al.  Recent advances in differential evolution: a survey and experimental analysis , 2010, Artificial Intelligence Review.

[35]  Enrique Alba,et al.  The exploration/exploitation tradeoff in dynamic cellular genetic algorithms , 2005, IEEE Transactions on Evolutionary Computation.

[36]  Teresa Wu,et al.  An Adaptive Particle Swarm Optimization With Multiple Adaptive Methods , 2013, IEEE Transactions on Evolutionary Computation.

[37]  H. Kushner.  Dynamical equations for optimal nonlinear filtering , 1967 .

[38]  Carolyn Kieran,et al.  Survey of the State of the Art , 2016 .

[39]  W. Wenzel,et al.  Stochastic Tunneling Approach for Global Minimization of Complex Potential Energy Landscapes , 1999 .

[40]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[41]  Changhe Li,et al.  A Self-Learning Particle Swarm Optimizer for Global Optimization Problems , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[42]  Mauro Birattari,et al.  Dm63 Heuristics for Combinatorial Optimization Ant Colony Optimization Exercises Outline Ant Colony Optimization: the Metaheuristic Application Examples Generalized Assignment Problem (gap) Connection between Aco and Other Metaheuristics Encodings Capacited Vehicle Routing Linear Ordering Ant Colony , 2022 .

[43]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[44]  Ponnuthurai Nagaratnam Suganthan,et al.  Benchmark Functions for the CEC'2013 Special Session and Competition on Large-Scale Global Optimization , 2008 .

[45]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[46]  Jasper A Vrugt,et al.  Improved evolutionary optimization from genetically adaptive multimethod search , 2007, Proceedings of the National Academy of Sciences.