p-MEMPSODE: Parallel and irregular memetic global optimization

Abstract A parallel memetic global optimization algorithm suitable for shared memory multicore systems is proposed and analyzed. The considered algorithm combines two well-known and widely used population-based stochastic algorithms, namely Particle Swarm Optimization and Differential Evolution, with two efficient and parallelizable local search procedures. The sequential version of the algorithm was first introduced as MEMPSODE (MEMetic Particle Swarm Optimization and Differential Evolution) and published in the CPC program library. We exploit the inherent and highly irregular parallelism of the memetic global optimization algorithm by means of a dynamic and multilevel approach based on the OpenMP tasking model. In our case, tasks correspond to local optimization procedures or simple function evaluations. Parallelization occurs at each iteration step of the memetic algorithm without affecting its searching efficiency. The proposed implementation, for the same random seed, reaches the same solution irrespectively of being executed sequentially or in parallel. Extensive experimental evaluation has been performed in order to illustrate the speedup achieved on a shared-memory multicore server. Program summary Program title: p-MEMPSODE Catalogue identifier: AEXJ_v1_0 Program summary URL: http://cpc.cs.qub.ac.uk/summaries/AEXJ_v1_0.html Program obtainable from: CPC Program Library, Queen’s University, Belfast, N. Ireland Licensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.html No. of lines in distributed program, including test data, etc.: 9950 No. of bytes in distributed program, including test data, etc.: 141503 Distribution format: tar.gz Programming language: ANSI C. Computer: Workstation. Operating system: Developed under the Linux operating system using the GNU compilers v.4.4.3 (or higher). Uses the OpenMP API and runtime system. RAM: The code uses O ( n × N ) internal storage, n being the dimension of the problem and N the maximum population size. The required memory is dynamically allocated. Word size: 64 Classification: 4.9. Nature of problem: Numerical global optimization of real valued functions is an indispensable methodology for solving a multitude of problems in science and engineering. Many problems exhibit a number of local and/or global minimizers, expensive function evaluations or require real-time response. In addition, discontinuities of the objective function, non-smooth and deceitful landscapes constitute challenging obstacles for most optimization algorithms. Solution method: We implement a memetic global optimization algorithm that combines stochastic, population-based methods with deterministic local search procedures. More specifically, the Unified Particle Swarm Optimization and the Differential Evolution algorithms are harnessed with the derivative-free Torczon’s Multi-Directional Search and the gradient-based BFGS method. The produced hybrid algorithms possess inherent parallelism that is exploited efficiently by means of the OpenMP tasking model. Given the same random seed, the proposed implementation reaches the same solution irrespective of being executed sequentially or in parallel. Restrictions: The current version of the software uses only double precision arithmetic. An OpenMP-enabled (version 3.0 or higher) compiler is required. Unusual features: The software requires bound constraints on the optimization variables. Running time: The running time depends on the complexity of the objective function (and its derivatives if used) as well as on the number of available cores. Extensive experimental results demonstrate that the speedup closely approximates ideal values.

[1]  Stephen R. Marsland,et al.  Convergence Properties of (μ + λ) Evolutionary Algorithms , 2011, AAAI.

[2]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[3]  Li Zheng,et al.  PGO: A parallel computing platform for global optimization based on genetic algorithm , 2007, Comput. Geosci..

[4]  Virginia Torczon,et al.  On the Convergence of the Multidirectional Search Algorithm , 1991, SIAM J. Optim..

[5]  Masha Sosonkina,et al.  Remark on Algorithm 897 , 2015 .

[6]  Konstantinos E. Parsopoulos,et al.  UPSO: A Unified Particle Swarm Optimization Scheme , 2019, International Conference of Computational Methods in Sciences and Engineering 2004 (ICCMSE 2004).

[7]  Isaac E. Lagaris,et al.  Particle swarm optimization with deliberate loss of information , 2012, Soft Comput..

[8]  Pablo Moscato,et al.  Memetic algorithms: a short introduction , 1999 .

[9]  R. Belew,et al.  Evolutionary algorithms with local search for combinatorial optimization , 1998 .

[10]  Pablo Moscato,et al.  Handbook of Memetic Algorithms , 2011, Studies in Computational Intelligence.

[11]  Michael N. Vrahatis,et al.  Particle Swarm Optimization and Intelligence: Advances and Applications , 2010 .

[12]  Juan J. Alonso,et al.  Aircraft design optimization , 2009, Math. Comput. Simul..

[13]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[14]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[15]  P. Kollman,et al.  A Second Generation Force Field for the Simulation of Proteins, Nucleic Acids, and Organic Molecules J. Am. Chem. Soc. 1995, 117, 5179−5197 , 1996 .

[16]  Christian L. Müller,et al.  pCMALib: a parallel fortran 90 library for the evolution strategy with covariance matrix adaptation , 2009, GECCO '09.

[17]  A. Dickson.  On Evolution , 1884, Science.

[18]  P. Pardalos,et al.  Handbook of global optimization , 1995 .

[19]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[20]  P. Kollman,et al.  A Second Generation Force Field for the Simulation of Proteins, Nucleic Acids, and Organic Molecules , 1995 .

[21]  Dimitris G. Papageorgiou,et al.  MEMPSODE: comparing particle swarm optimization and differential evolution within a hybrid memetic global optimization framework , 2012, GECCO '12.

[22]  W. Hart.  Adaptive global optimization with local search , 1994 .

[23]  D. Fogel.  Evolutionary algorithms in theory and practice , 1997, Complex..

[24]  Dimitris G. Papageorgiou,et al.  MEMPSODE: an empirical assessment of local search algorithm impact on a memetic algorithm using noiseless testbed , 2012, GECCO '12.

[25]  Michael N. Vrahatis,et al.  Memetic particle swarm optimization , 2007, Ann. Oper. Res..

[26]  Michael N. Vrahatis,et al.  Parameter selection and adaptation in Unified Particle Swarm Optimization , 2007, Math. Comput. Model..

[27]  K. V. Price,et al.  Differential evolution: a fast and simple numerical optimizer , 1996, Proceedings of North American Fuzzy Information Processing.

[28]  Dario Izzo,et al.  The Generalized Island Model , 2012, Parallel Architectures and Bioinspired Algorithms.

[29]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[30]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[31]  Suganthan.  [IEEE 1999. Congress on Evolutionary Computation-CEC99 - Washington, DC, USA (6-9 July 1999)] Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406) - Particle swarm optimiser with neighbourhood operator , 1999 .

[32]  Andries Petrus Engelbrecht,et al.  Fundamentals of Computational Swarm Intelligence , 2005 .

[33]  V. J. Torczoit,et al.  Multidirectional search: a direct search algorithm for parallel machines , 1989 .

[34]  Constantinos Voglis,et al.  Adapt-MEMPSODE: a memetic algorithm with adaptive selection of local searches , 2013, GECCO.

[35]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[36]  R. Lewontin.  ‘The Selfish Gene’ , 1977, Nature.

[37]  J. Betts.  Survey of Numerical Methods for Trajectory Optimization , 1998 .

[38]  Dimitris G. Papageorgiou,et al.  A numerical differentiation library exploiting parallel architectures ✩ , 2009 .

[39]  L. Girifalco.  Molecular properties of fullerene in the gas and solid phases , 1992 .

[40]  Marco Dorigo,et al.  Swarm intelligence: from natural to artificial systems , 1999 .

[41]  Michael N. Vrahatis,et al.  MEMPSODE: A global optimization software based on hybridization of population-based algorithms and local searches , 2012, Comput. Phys. Commun..

[42]  P. Morse.  Diatomic Molecules According to the Wave Mechanics. II. Vibrational Levels , 1929 .

[43]  Pablo Moscato,et al.  On Evolution, Search, Optimization, Genetic Algorithms and Martial Arts : Towards Memetic Algorithms , 1989 .

[44]  Dario Izzo,et al.  FOR MASSIVELY PARALLEL OPTIMIZATION IN ASTRODYNAMICS ( THE CASE OF INTERPLANETARY TRAJECTORY OPTIMIZATION ) , 2012 .

[45]  Philip E. Gill,et al.  Practical optimization , 1981 .

[46]  Dimitris G. Papageorgiou,et al.  MERLIN-3.1.1. A new version of the Merlin optimization environment , 2004 .

[47]  Antanas Zilinskas,et al.  A hybrid global optimization algorithm for non-linear least squares regression , 2012, Journal of Global Optimization.

[48]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[49]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[50]  Janet E. Jones.  On the determination of molecular fields. III.—From crystal measurements and kinetic theory data , 1924 .

[51]  Dimitris G. Papageorgiou,et al.  Adaptive memetic particle swarm optimization with variable local search pool size , 2013, GECCO '13.

[52]  H. Beyer.  Evolutionary algorithms in noisy environments : theoretical issues and guidelines for practice , 2000 .

[53]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..