Enhanced Particle Swarm Optimization Based on Reference Direction and Inverse Model for Optimization Problems

While particle swarm optimization (PSO) shows good performance for many optimization problems, the weakness in premature convergence and easy trapping into local optimum, due to the ignorance of the diversity information, has been gradually recognized. To improve the optimization performance of PSO, an enhanced PSO based on reference direction and inverse model is proposed, RDIM-PSO for short reference. In RDIM-PSO, the reference particles which are used as reference directions are selected by non-dominated sorting method according to the fitness and diversity contribution of the population. Dynamic neighborhood strategy is introduced to divide the population into several sub-swarms based on the reference directions. For each sub-swarm, the particles focus on exploitation under the guidance of local best particle with a good guarantee of population diversity. Moreover, Gaussian process-based inverse model is introduced to generate equilibrium particles by sampling the objective space to further achieve a good balance between exploration and exploitation. Experimental results on CEC2014 test problems show that RDIM-PSO has overall better performance compared with other well-known optimization algorithms. Finally, the proposed RDIM-PSO is also applied to artificial neural networks and the promising results on the chaotic time series prediction show the effectiveness of RDIM-PSO.

[1]  Jing J. Liang,et al.  Comprehensive learning particle swarm optimizer for global optimization of multimodal functions , 2006, IEEE Transactions on Evolutionary Computation.

[2]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[3]  J. Miller,et al.  Guidelines: From artificial evolution to computational evolution: a research agenda , 2006, Nature Reviews Genetics.

[4]  R. Rao Jaya: A simple and new optimization algorithm for solving constrained and unconstrained optimization problems , 2016 .

[5]  Wei-Der Chang,et al.  A modified particle swarm optimization with multiple subpopulations for multimodal function optimization problems , 2015, Appl. Soft Comput..

[6]  R. Venkata Rao,et al.  Teaching-Learning-Based Optimization: An optimization method for continuous non-linear large scale problems , 2012, Inf. Sci..

[7]  Paulo Cortez,et al.  Particle swarms for feedforward neural network training , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[8]  Yaochu Jin,et al.  A social learning particle swarm optimization algorithm for scalable optimization , 2015, Inf. Sci..

[9]  P. N. Suganthan,et al.  A dynamic neighborhood learning based particle swarm optimizer for global numerical optimization , 2012, Inf. Sci..

[10]  Shengxiang Yang,et al.  A Strength Pareto Evolutionary Algorithm Based on Reference Direction for Multiobjective and Many-Objective Optimization , 2017, IEEE Transactions on Evolutionary Computation.

[11]  Vasilii A. Gromov,et al.  Chaotic time series prediction with employment of ant colony optimization , 2012, Expert Syst. Appl..

[12]  Qingfu Zhang,et al.  Differential Evolution With Composite Trial Vector Generation Strategies and Control Parameters , 2011, IEEE Transactions on Evolutionary Computation.

[13]  Xinchao Zhao,et al.  Clustering and pattern search for enhancing particle swarm optimization with Euclidean spatial neighborhood search , 2016, Neurocomputing.

[14]  Robert W. Cahn,et al.  Learning from nature , 2006, Nature.

[15]  Jianchao Zeng,et al.  Surrogate-Assisted Cooperative Swarm Optimization of High-Dimensional Expensive Problems , 2017, IEEE Transactions on Evolutionary Computation.

[16]  Yuhui Shi,et al.  Brain Storm Optimization Algorithm , 2011, ICSI.

[17]  Jun Zhang,et al.  Orthogonal Learning Particle Swarm Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[18]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[19]  Hui Wang,et al.  Diversity enhanced particle swarm optimization with neighborhood search , 2013, Inf. Sci..

[20]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[21]  Steven Li,et al.  Improved global-best-guided particle swarm optimization with learning operation for global optimization problems , 2017, Appl. Soft Comput..

[22]  Toshiharu Sugie,et al.  Synthesis of fixed-structure robust controllers using a constrained particle swarm optimizer with cyclic neighborhood topology , 2013, Expert Syst. Appl..

[23]  B. Samanta,et al.  Prediction of chaotic time series using computational intelligence , 2011, Expert Syst. Appl..

[24]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[25]  Wenxing Ye,et al.  A novel multi-swarm particle swarm optimization with dynamic learning strategy , 2017, Appl. Soft Comput..

[26]  Jeff Orchard,et al.  Particle swarm optimization using dynamic tournament topology , 2016, Appl. Soft Comput..

[27]  Dan Simon,et al.  Biogeography-Based Optimization , 2022 .

[28]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[29]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[30]  Amit Konar,et al.  Differential Evolution Using a Neighborhood-Based Mutation Operator , 2009, IEEE Transactions on Evolutionary Computation.

[31]  Ke Chen,et al.  Chaotic dynamic weight particle swarm optimization for numerical function optimization , 2018, Knowl. Based Syst..

[32]  Teresa Wu,et al.  An Adaptive Particle Swarm Optimization With Multiple Adaptive Methods , 2013, IEEE Transactions on Evolutionary Computation.

[33]  Yaochu Jin,et al.  A Competitive Swarm Optimizer for Large Scale Optimization , 2015, IEEE Transactions on Cybernetics.

[34]  Ying Tan,et al.  Fireworks Algorithm for Optimization , 2010, ICSI.

[35]  Udo Seiffert,et al.  Multiple Layer Perceptron training using genetic algorithms , 2001, ESANN.

[36]  Siti Zaiton Mohd Hashim,et al.  Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm , 2012, Appl. Math. Comput..

[37]  Gexiang Zhang,et al.  Enhancing distributed differential evolution with multicultural migration for global numerical optimization , 2013, Inf. Sci..

[38]  Victor O. K. Li,et al.  Chemical-Reaction-Inspired Metaheuristic for Optimization , 2010, IEEE Transactions on Evolutionary Computation.

[39]  Wei Sun,et al.  All-dimension neighborhood based particle swarm optimization with randomly selected neighbors , 2017, Inf. Sci..

[40]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[41]  Laizhong Cui,et al.  Adaptive differential evolution algorithm with novel mutation strategies in multiple sub-populations , 2016, Comput. Oper. Res..

[42]  Konstantinos Demertzis,et al.  Adaptive Elitist Differential Evolution Extreme Learning Machines on Big Data: Intelligent Recognition of Invasive Species , 2016, INNS Conference on Big Data.

[43]  José Neves,et al.  The fully informed particle swarm: simpler, maybe better , 2004, IEEE Transactions on Evolutionary Computation.

[44]  P. Suganthan Particle swarm optimiser with neighbourhood operator , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[45]  Yiqiao Cai,et al.  Differential Evolution Enhanced With Multiobjective Sorting-Based Mutation Operators , 2014, IEEE Transactions on Cybernetics.

[46]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[47]  María José del Jesús,et al.  KEEL: a software tool to assess evolutionary algorithms for data mining problems , 2008, Soft Comput..

[48]  Dervis Karaboga,et al.  A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm , 2007, J. Glob. Optim..

[49]  Christian Blum,et al.  Training feed-forward neural networks with ant colony optimization: an application to pattern classification , 2005, Fifth International Conference on Hybrid Intelligent Systems (HIS'05).

[50]  Andries Petrus Engelbrecht,et al.  A Cooperative approach to particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[51]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[52]  Andrew Lewis,et al.  The Whale Optimization Algorithm , 2016, Adv. Eng. Softw..

[53]  G. Wiselin Jiji,et al.  An enhanced particle swarm optimization with levy flight for global optimization , 2016, Appl. Soft Comput..

[54]  Liang Zhao,et al.  PSO-based single multiplicative neuron model for time series prediction , 2009, Expert Syst. Appl..

[55]  Bernhard Sendhoff,et al.  A Multiobjective Evolutionary Algorithm Using Gaussian Process-Based Inverse Modeling , 2015, IEEE Transactions on Evolutionary Computation.

[56]  Narasimhan Sundararajan,et al.  Self regulating particle swarm optimization algorithm , 2015, Inf. Sci..