How to design a powerful family of particle swarm optimizers for inverse modelling

In this paper, we show how to design a powerful set of particle swarm optimizers to be applied in inverse modelling. The design is based on the interpretation of the swarm dynamics as a stochastic damped mass-spring system, the so-called particle swarm optimization (PSO) continuous model. Based on this idea we derived a family of PSO optimizers (GPSO, CC-PSO and CP-PSO) having different exploitation and exploration capabilities. Their convergence is related to the stability of their first (mean trajectories)- and second-order moments (variance and temporal covariance). Good parameter sets are located inside their first stability regions close to the upper border of their respective second stability regions where the attraction from the particles oscillation centre is very weak. In this region of weak attraction, both convergence to the global minimum and exploration of the search space are possible. Based on this idea, we have designed a particle–cloud algorithm where each particle in the swarm has different inertia (damping) and acceleration (rigidity) constants. We explored the performance of these algorithms for different PSO members using different benchmark functions, showing that the cloud algorithms have a very good balance between exploration and exploitation. Also, the cloud design helps to avoid two main drawbacks of the PSO algorithm: the tuning of the PSO parameters and the clamping of the particles velocities. We also present the lime and sand algorithm that changes the time step with iterations. This feature helps to avoid entrapment in local minima when the time step is increased, and enables exploration around the global best when the time step is decreased. All these designs are based on the theoretical analysis of the PSO dynamics. We explain how to use this knowledge to the solution and appraisal of inverse problems. Finally, we briefly introduce the combined use of PSO and model reduction techniques to allow posterior sampling in high dimensional spaces.

[1]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[2]  Russell C. Eberhart,et al.  Comparison between Genetic Algorithms and Particle Swarm Optimization , 1998, Evolutionary Programming.

[3]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[4]  Russell C. Eberhart,et al.  Parameter Selection in Particle Swarm Optimization , 1998, Evolutionary Programming.

[5]  James Kennedy,et al.  The Behavior of Particles , 1998, Evolutionary Programming.

[6]  E. Ozcan,et al.  Particle swarm optimization: surfing the waves , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[7]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[8]  R. Eberhart,et al.  Fuzzy adaptive particle swarm optimization , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[9]  Vassilis P. Plagianakos,et al.  Improving the Particle Swarm Optimizer by Function “Stretching” , 2001 .

[10]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[11]  Brian Birge,et al.  PSOt - a particle swarm optimization toolbox for use with Matlab , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[12]  Liyan Zhang,et al.  Empirical study of particle swarm optimizer with an increasing inertia weight , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[13]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[14]  Yongling Zheng,et al.  On the convergence analysis and parameter selection in particle swarm optimization , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[15]  Michael N. Vrahatis,et al.  On the computation of all global minimizers through particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[16]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[17]  Niels Bohr,et al.  Monte Carlo sampling of solutions to inverse problems , 2004 .

[18]  Zhang Li-ping,et al.  Optimal choice of parameters for particle swarm optimization , 2005 .

[19]  Andries Petrus Engelbrecht,et al.  A study of particle swarm optimization particle trajectories , 2006, Inf. Sci..

[20]  Z. Cui,et al.  A FAST PARTICLE SWARM OPTIMIZATION , 2006 .

[21]  Shiyuan Yang,et al.  Stochastic convergence analysis and parameter selection of the standard particle swarm optimization algorithm , 2007, Inf. Process. Lett..

[22]  M. Jiang,et al.  Particle Swarm Optimization - Stochastic Trajectory Analysis and Parameter Selection , 2007 .

[23]  Shiyuan Yang,et al.  Stagnation Analysis in Particle Swarm Optimization , 2007, 2007 IEEE Swarm Intelligence Symposium.

[24]  Juan Luis Fernández-Martínez,et al.  Theoretical analysis of particle swarm trajectories through a mechanical analogy , 2008 .

[25]  J. F. Martínez,et al.  The generalized PSO: a new door to PSO evolution , 2008 .

[26]  J. P. Fernández Alvarez,et al.  Feasibility Analysis of the Use of Binary Genetic Algorithms as Importance Samplers Application to a 1-D DC Resistivity Inverse Problem , 2008 .

[27]  Zhihua Cui,et al.  Particle swarm optimization with FUSS and RWS for high dimensional functions , 2008, Appl. Math. Comput..

[28]  Riccardo Poli,et al.  Dynamics and stability of the sampling distribution of particle swarm optimisers via moment analysis , 2008 .

[29]  Juan Luis Fernández-Martínez,et al.  Application of Global Optimization Algorithms to a Salt Water Intrusion Problem , 2009 .

[30]  Esperanza García Gonzalo,et al.  The PSO family: deduction, stochastic analysis and comparison , 2009, Swarm Intelligence.

[31]  Ying Tan,et al.  A study on the effect of vmax in particle swarm optimisation with high dimension , 2009, Int. J. Bio Inspired Comput..

[32]  Srinivasan Alavandar,et al.  A hybrid genetically-bacterial foraging algorithm converged by particle swarm optimisation for global optimisation , 2010, Int. J. Bio Inspired Comput..

[33]  K. Gomathy,et al.  Damping of low frequency oscillations in power system network using swarm intelligence tuned fuzzy controller , 2010, Int. J. Bio Inspired Comput..

[34]  Fan Gu,et al.  Robust decoupling control of BTT vehicle based on PSO , 2010, Int. J. Bio Inspired Comput..

[35]  J. Fernández-Martínez,et al.  Particle swarm optimization applied to solving and appraising the streaming-potential inverse problem , 2010 .

[36]  Sugata Sanyal,et al.  To filter or to authorize , 2008 .

[37]  Hong Yang,et al.  Improved strategy of particle swarm optimisation algorithm for reactive power optimisation , 2010, Int. J. Bio Inspired Comput..

[38]  Esperanza García Gonzalo,et al.  PSO: A powerful algorithm to solve geophysical inverse problems: Application to a 1D-DC resistivity case , 2010 .

[39]  E. G. Gonzalo,et al.  Stochastic Stability Analysis of the Linear Continuous and Discrete PSO Models , 2011, IEEE Trans. Evol. Comput..

[40]  J. Fernández-Martínez,et al.  Stochastic Stability Analysis of the Linear Continuous and Discrete PSO Models , 2011, IEEE Transactions on Evolutionary Computation.

[41]  Josep Lluís Carrion‐i‐Silvestre,et al.  Stochastic Convergence Analysis of the Mexican Regions , 2014 .