Multi-Strategy coevolving aging Particle Optimization

We propose Multi-Strategy Coevolving Aging Particles (MS-CAP), a novel population-based algorithm for black-box optimization. In a memetic fashion, MS-CAP combines two components with complementary algorithm logics. In the first stage, each particle is perturbed independently along each dimension with a progressively shrinking (decaying) radius, and attracted towards the current best solution with an increasing force. In the second phase, the particles are mutated and recombined according to a multi-strategy approach in the fashion of the ensemble of mutation strategies in Differential Evolution. The proposed algorithm is tested, at different dimensionalities, on two complete black-box optimization benchmarks proposed at the Congress on Evolutionary Computation 2010 and 2013. To demonstrate the applicability of the approach, we also test MS-CAP to train a Feedforward Neural Network modeling the kinematics of an 8-link robot manipulator. The numerical results show that MS-CAP, for the setting considered in this study, tends to outperform the state-of-the-art optimization algorithms on a large set of problems, thus resulting in a robust and versatile optimizer.

[1]  Hojjat Adeli,et al.  High-Performance Computing for Large-Scale Analysis, Optimization, and Control , 2000 .

[2]  Ponnuthurai N. Suganthan,et al.  An Adaptive Differential Evolution Algorithm With Novel Mutation and Crossover Strategies for Global Numerical Optimization , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[3]  Rosario Baltazar,et al.  Comparison of PSO and DE for Training Neural Networks , 2011, 2011 10th Mexican International Conference on Artificial Intelligence.

[4]  Dimitris C. Theodoridis,et al.  Dynamical Recurrent Neuro-Fuzzy Identification Schemes Employing Switching parameter Hopping , 2012, Int. J. Neural Syst..

[5]  Pablo Moscato,et al.  Handbook of Memetic Algorithms , 2011, Studies in Computational Intelligence.

[6]  Francisco Herrera,et al.  A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability , 2009, Soft Comput..

[7]  María Dolores Rodríguez-Moreno,et al.  Acquisition of business intelligence from human experience in route planning , 2015, Enterp. Inf. Syst..

[8]  Hojjat Adeli,et al.  Enhanced probabilistic neural network with local decision circles: A robust classifier , 2010, Integr. Comput. Aided Eng..

[9]  James A. Reggia,et al.  Causally-guided evolutionary optimization and its application to antenna array design , 2012, Integr. Comput. Aided Eng..

[10]  Carlos García-Martínez,et al.  Memetic Algorithms for Continuous Optimisation Based on Local Search Chains , 2010, Evolutionary Computation.

[11]  Anikó Ekárt,et al.  Genetic algorithms in computer aided design , 2003, Comput. Aided Des..

[12]  Ferrante Neri,et al.  Memetic Compact Differential Evolution for Cartesian Robot Control , 2010, IEEE Computational Intelligence Magazine.

[13]  Hichem Maaref,et al.  Two inertial models of X4-flyers dynamics, motion planning and control , 2007, Integr. Comput. Aided Eng..

[14]  Hojjat Adeli,et al.  A synergic man-machine approach to shape optimization of structures , 1988 .

[15]  Carlos Cotta,et al.  Memetic algorithms and memetic computing optimization: A literature review , 2012, Swarm Evol. Comput..

[16]  Carlos Cotta,et al.  Recent Advances in Evolutionary Computation for Combinatorial Optimization , 2008, Recent Advances in Evolutionary Computation for Combinatorial Optimization.

[17]  N. Burum,et al.  Using particle swarm optimization in training neural network for indoor field strength prediction , 2009, 2009 International Symposium ELMAR.

[18]  Xin Yao,et al.  An Experimental Study of Hybridizing Cultural Algorithms and Local Search , 2008, Int. J. Neural Syst..

[19]  Jing J. Liang,et al.  Comprehensive learning particle swarm optimizer for global optimization of multimodal functions , 2006, IEEE Transactions on Evolutionary Computation.

[20]  Manolis Papadrakakis,et al.  A Hybrid Particle Swarm—Gradient Algorithm for Global Structural Optimization , 2010, Comput. Aided Civ. Infrastructure Eng..

[21]  William E. Hart,et al.  Recent Advances in Memetic Algorithms , 2008 .

[22]  Xin Chen,et al.  A New Stochastic PSO Technique for Neural Network Training , 2006, ISNN.

[23]  Ville Tirronen,et al.  Recent advances in differential evolution: a survey and experimental analysis , 2010, Artificial Intelligence Review.

[24]  Natalio Krasnogor,et al.  Towards Robust Memetic Algorithms , 2005 .

[25]  Hojjat Adeli,et al.  Integrated structural/control optimization of large adaptive/smart structures , 1998 .

[26]  Silvia Tolu,et al.  Adaptive and Predictive Control of a Simulated Robot arm , 2013, Int. J. Neural Syst..

[27]  Brian Moran,et al.  A finite element formulation for transient analysis of viscoplastic solids with application to stress wave propagation problems , 1987 .

[28]  Arthur C. Sanderson,et al.  JADE: Adaptive Differential Evolution With Optional External Archive , 2009, IEEE Transactions on Evolutionary Computation.

[29]  Hojjat Adeli,et al.  Efficient optimization of space trusses , 1986 .

[30]  G. Ghodrati Amiri,et al.  Generation of Near‐Field Artificial Ground Motions Compatible with Median‐Predicted Spectra Using PSO‐Based Neural Network and Wavelet Analysis , 2012, Comput. Aided Civ. Infrastructure Eng..

[31]  Pablo Moscato,et al.  On Evolution, Search, Optimization, Genetic Algorithms and Martial Arts : Towards Memetic Algorithms , 1989 .

[32]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[33]  Meng Joo Er,et al.  A Novel Efficient Learning Algorithm for Self-Generating Fuzzy Neural Network with Applications , 2012, Int. J. Neural Syst..

[34]  Hojjat Adeli,et al.  A probabilistic neural network for earthquake magnitude prediction , 2009, Neural Networks.

[35]  S. Holm A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[36]  Adam Prügel-Bennett,et al.  Benefits of a Population: Five Mechanisms That Advantage Population-Based Algorithms , 2010, IEEE Transactions on Evolutionary Computation.

[37]  Hai-Bin Duan,et al.  A Hybrid Artificial Bee Colony Optimization and Quantum Evolutionary Algorithm for Continuous Optimization Problems , 2010, Int. J. Neural Syst..

[38]  Marc M. Van Hulle,et al.  Enhancing the Yield of High-Density electrode Arrays through Automated electrode Selection , 2012, Int. J. Neural Syst..

[39]  Francisco Herrera,et al.  MA-SW-Chains: Memetic algorithm based on local search chains for large scale continuous global optimization , 2010, IEEE Congress on Evolutionary Computation.

[40]  Giovanni Iacca,et al.  Disturbed Exploitation compact Differential Evolution for limited memory optimization problems , 2011, Inf. Sci..

[41]  Kamal C. Sarma,et al.  FUZZY GENETIC ALGORITHM FOR OPTIMIZATION OF STEEL STRUCTURES , 2000 .

[42]  Giovanni Iacca,et al.  Ockham's Razor in memetic computing: Three stage optimal memetic exploration , 2012, Inf. Sci..

[43]  Hojjat Adeli,et al.  Optimization of hybrid steel plate girders , 1987 .

[44]  Rudolf Scitovski,et al.  Solving the parameter identification problem of mathematical models using genetic algorithms , 2004, Appl. Math. Comput..

[45]  Raymond Ros,et al.  A Simple Modification in CMA-ES Achieving Linear Time and Space Complexity , 2008, PPSN.

[46]  Xiaodong Li,et al.  Benchmark Functions for the CEC'2010 Special Session and Competition on Large-Scale , 2009 .

[47]  Giovanni Iacca,et al.  Parallel memetic structures , 2013, Inf. Sci..

[48]  Luca Quadrifoglio,et al.  Comparing Ant Colony Optimization and Genetic Algorithm Approaches for Solving Traffic Signal Coordination under Oversaturation Conditions , 2012, Comput. Aided Civ. Infrastructure Eng..

[49]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[50]  Janez Brest,et al.  Self-Adapting Control Parameters in Differential Evolution: A Comparative Study on Numerical Benchmark Problems , 2006, IEEE Transactions on Evolutionary Computation.

[51]  Ponnuthurai N. Suganthan,et al.  Differential Evolution Algorithm with Ensemble of Parameters and Mutation and Crossover Strategies , 2010, SEMCCO.

[52]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[53]  María Dolores Rodríguez-Moreno,et al.  A Decision Support System for Logistics Operations , 2010, SOCO.

[54]  Ying Lin,et al.  Particle Swarm Optimization With an Aging Leader and Challengers , 2013, IEEE Transactions on Evolutionary Computation.

[55]  Qingfu Zhang,et al.  A Multiobjective Differential Evolution Based on Decomposition for Multiobjective Optimization with Variable Linkages , 2006, PPSN.

[56]  Silvia Tolu,et al.  Adaptive cerebellar Spiking Model Embedded in the Control Loop: Context Switching and Robustness against noise , 2011, Int. J. Neural Syst..

[57]  Jeff Heaton,et al.  Programming Neural Networks with Encog 2 in Java , 2010 .

[58]  Ville Tirronen,et al.  Super-fit control adaptation in memetic differential evolution frameworks , 2009, Soft Comput..

[59]  Jing Wang,et al.  A wavelet-based particle swarm optimization algorithm for digital image watermarking , 2012, Integr. Comput. Aided Eng..

[60]  P. N. Suganthan,et al.  Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[61]  Joni-Kristian Kämäräinen,et al.  Differential Evolution Training Algorithm for Feed-Forward Neural Networks , 2003, Neural Processing Letters.

[62]  Francisco Herrera,et al.  Editorial scalability of evolutionary algorithms and other metaheuristics for large-scale continuous optimization problems , 2011, Soft Comput..

[63]  Welch Bl THE GENERALIZATION OF ‘STUDENT'S’ PROBLEM WHEN SEVERAL DIFFERENT POPULATION VARLANCES ARE INVOLVED , 1947 .

[64]  Peter I. Corke,et al.  A robotics toolbox for MATLAB , 1996, IEEE Robotics Autom. Mag..

[65]  Giovanni Iacca,et al.  Memory-saving memetic computing for path-following mobile robots , 2013, Appl. Soft Comput..

[66]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[67]  Rob Law,et al.  Complex system fault diagnosis based on a fuzzy robust wavelet support vector classifier and an adaptive Gaussian particle swarm optimization , 2010, Inf. Sci..

[68]  David Camacho,et al.  Adaptive k-Means Algorithm for Overlapped Graph Clustering , 2012, Int. J. Neural Syst..

[69]  Hojjat Adeli,et al.  Probabilistic neural networks for diagnosis of Alzheimer's disease using conventional and wavelet coherence , 2011, Journal of Neuroscience Methods.

[70]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[71]  Huang Hou-kuan Self-adapting control parameters in differential evolution , 2012 .

[72]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[73]  Petros Koumoutsakos,et al.  Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES) , 2003, Evolutionary Computation.

[74]  Geoffrey Bower,et al.  MULTI-OBJECTIVE AIRCRAFT OPTIMIZATION FOR MINIMUM COST AND EMISSIONS OVER SPECIFIC ROUTE NETWORKS , 2008 .

[75]  Juan Humberto Sossa Azuela,et al.  Evolving Neural Networks: A Comparison between Differential Evolution and Particle Swarm Optimization , 2011, ICSI.

[76]  Jing Wang,et al.  A novel particle swarm algorithm for solving parameter identification problems on graphics hardware , 2011, Int. J. Comput. Sci. Eng..

[77]  M. Bialko,et al.  Training of artificial neural networks using differential evolution algorithm , 2008, 2008 Conference on Human System Interactions.

[78]  Ville Tirronen,et al.  Scale factor inheritance mechanism in distributed differential evolution , 2009, Soft Comput..

[79]  Niko Kotilainen,et al.  A Memetic-Neural Approach to Discover Resources in P2P Networks , 2008, Recent Advances in Evolutionary Computation for Combinatorial Optimization.

[80]  Hisao Ishibuchi,et al.  Balance between genetic search and local search in memetic algorithms for multiobjective permutation flowshop scheduling , 2003, IEEE Trans. Evol. Comput..

[81]  Xiaodong Li,et al.  Cooperatively Coevolving Particle Swarms for Large Scale Optimization , 2012, IEEE Transactions on Evolutionary Computation.

[82]  Eduardo Ros,et al.  From Sensors to Spikes: Evolving Receptive Fields to Enhance Sensorimotor Information in a Robot-Arm , 2012, Int. J. Neural Syst..

[83]  Student,et al.  THE PROBABLE ERROR OF A MEAN , 1908 .

[84]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[85]  Mehmet Fatih Tasgetiren,et al.  Differential evolution algorithm with ensemble of parameters and mutation strategies , 2011, Appl. Soft Comput..

[86]  J. Denavit,et al.  A kinematic notation for lower pair mechanisms based on matrices , 1955 .