Global optimization using a multipoint type quasi-chaotic optimization method

This paper proposes a new global optimization method called the multipoint type quasi-chaotic optimization method. In the proposed method, the simultaneous perturbation gradient approximation is introduced into a multipoint type chaotic optimization method in order to carry out optimization without gradient information. The multipoint type chaotic optimization method, which has been proposed recently, is a global optimization method for solving unconstrained optimization problems in which multiple search points which implement global searches driven by a chaotic gradient dynamic model are advected to their elite search points (best search points among the current search histories). The chaotic optimization method uses a gradient to drive search points. Hence, its application is restricted to a class of problems in which the gradient of the objective function can be computed. In this paper, the simultaneous perturbation gradient approximation is introduced into the multipoint type chaotic optimization method in order to approximate gradients so that the chaotic optimization method can be applied to a class of problems for which only the objective function values can be computed. Then, the effectiveness of the proposed method is confirmed through application to several unconstrained multi-peaked, noisy, or discontinuous optimization problems with 100 or more variables, comparing to other major meta-heuristics.

[1]  L. Gerencsér,et al.  SPSA in noise free optimization , 2000, Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No.00CH36334).

[2]  Wenyin Gong,et al.  DE/BBO: a hybrid differential evolution with biogeography-based optimization for global numerical optimization , 2010, Soft Comput..

[3]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[4]  J. L. Maryak,et al.  P Global Random Optimization by Simultaneous Perturbation Stochastic Approximation , 2005 .

[5]  J. L. Maryak,et al.  Simultaneous perturbation optimization for efficient image restoration , 2005, IEEE Transactions on Aerospace and Electronic Systems.

[6]  Takashi Okamoto,et al.  Global optimization using a synchronization of multiple search Points autonomously driven by a chaotic dynamic model , 2008, J. Glob. Optim..

[7]  Moncef Gabbouj,et al.  Stochastic approximation driven particle swarm optimization with simultaneous perturbation - Who will guide the guide? , 2011, Appl. Soft Comput..

[8]  Han-Fu Chen,et al.  A Stochastic Approximation Algorithm with Random Differences , 1996 .

[9]  James C. Spall,et al.  Introduction to stochastic search and optimization - estimation, simulation, and control , 2003, Wiley-Interscience series in discrete mathematics and optimization.

[10]  Michael C. Fu,et al.  Convergence of simultaneous perturbation stochastic approximation for nondifferentiable optimization , 2003, IEEE Trans. Autom. Control..

[11]  Moncef Gabbouj,et al.  Fractional Particle Swarm Optimization in Multidimensional Search Space , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[12]  Masato Mizukami,et al.  Simultaneous alignment of multiple optical axes in a multistage optical system using Hamiltonian algorithm , 2001 .

[13]  Yutaka Maeda,et al.  Simultaneous perturbation learning rule for recurrent neural networks and its FPGA implementation , 2005, IEEE Transactions on Neural Networks.

[14]  Kazuyuki Aihara,et al.  Global bifurcation scenario for chaotic dynamical systems that solve optimization problems and analysis of their optimization capability , 1998 .

[15]  James C. Spall,et al.  Introduction to Stochastic Search and Optimization. Estimation, Simulation, and Control (Spall, J.C. , 2007 .

[16]  S. Bhatnagar.  Simultaneous Perturbation and Finite Difference Methods , 2011 .

[17]  Jing J. Liang,et al.  Performance Evaluation of Multiagent Genetic Algorithm , 2006, Natural Computing.

[18]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[19]  Nusawardhana,et al.  Simultaneous perturbation extremum seeking method for dynamic optimization problems , 2004, Proceedings of the 2004 American Control Conference.

[20]  Jing J. Liang,et al.  Novel composition test functions for numerical global optimization , 2005, Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005..

[21]  K. Aihara,et al.  Chaotic neural networks , 1990 .

[22]  Jürgen Kurths,et al.  Synchronization - A Universal Concept in Nonlinear Sciences , 2001, Cambridge Nonlinear Science Series.

[23]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .