Particle Swarm Optimization: A Tutorial

Optimization algorithms are necessary to solve many problems such as parameter tuning. Particle Swarm Optimization (PSO) is one of these optimization algorithms. The aim of PSO is to search for the optimal solution in the search space. This paper highlights the basic background needed to understand and implement the PSO algorithm. This paper starts with basic definitions of the PSO algorithm and how the particles are moved in the search space to find the optimal or near optimal solution. Moreover, a numerical example is illustrated to show how the particles are moved in a convex optimization problem. Another numerical example is illustrated to show how the PSO trapped in a local minima problem. Two experiments are conducted to show how the PSO searches for the optimal parameters in one-dimensional and two-dimensional spaces to solve machine learning problems.

[1]  Frans van den Bergh,et al.  An analysis of particle swarm optimizers , 2002 .

[2]  Alaa Tharwat,et al.  Principal component analysis - a tutorial , 2016, Int. J. Appl. Pattern Recognit..

[3]  Zhao Xinchao.  A perturbed particle swarm algorithm for numerical optimization , 2010 .

[4]  Iwao Okamoto,et al.  Magnetic interaction in Co-Cr-Pt-Ta-Nb media: utilization of micromagnetic simulation , 1998 .

[5]  Bahriye Akay,et al.  A study on particle swarm optimization and artificial bee colony algorithms for multilevel thresholding , 2013, Appl. Soft Comput..

[6]  Gerald Schaefer,et al.  CT Liver Segmentation Using Artificial Bee Colony Optimisation , 2015, KES.

[7]  Aboul Ella Hassanien,et al.  Detection of heart disease using binary particle swarm optimization , 2012, 2012 Federated Conference on Computer Science and Information Systems (FedCSIS).

[8]  Shih-Wei Lin,et al.  Particle swarm optimization for parameter determination and feature selection of support vector machines , 2008, Expert Syst. Appl..

[9]  Z.A. Bashir,et al.  Applying Wavelets to Short-Term Load Forecasting Using PSO-Based Neural Networks , 2009, IEEE Transactions on Power Systems.

[10]  O. Weck,et al.  A COMPARISON OF PARTICLE SWARM OPTIMIZATION AND THE GENETIC ALGORITHM , 2005 .

[11]  Václav Snásel,et al.  Biometric cattle identification approach based on Weber's Local Descriptor and AdaBoost classifier , 2016, Comput. Electron. Agric..

[12]  Aboul Ella Hassanien,et al.  A BA-based algorithm for parameter optimization of Support Vector Machine , 2017, Pattern Recognit. Lett..

[13]  Amitava Chatterjee,et al.  A new social and momentum component adaptive PSO algorithm for image segmentation , 2011, Expert Syst. Appl..

[14]  Andries Petrus Engelbrecht,et al.  Data clustering using particle swarm optimization , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[15]  Aboul Ella Hassanien,et al.  One-dimensional vs. two-dimensional based features: Plant identification approach , 2017, J. Appl. Log..

[16]  Aboul Ella Hassanien,et al.  A New Multi-layer Perceptrons Trainer Based on Ant Lion Optimization Algorithm , 2015, 2015 Fourth International Conference on Information Science and Industrial Applications (ISI).

[17]  S.Y. Yang,et al.  An Improved PSO Method With Application to Multimodal Functions of Inverse Problems , 2007, IEEE Transactions on Magnetics.

[18]  Amitava Chatterjee,et al.  A hybrid cooperative-comprehensive learning based PSO algorithm for image segmentation using multilevel thresholding , 2008, Expert Syst. Appl..

[19]  Aboul Ella Hassanien,et al.  Automated zebrafish-based toxicity test using Bat optimization and AdaBoost classifier , 2015, 2015 11th International Computer Engineering Conference (ICENCO).

[20]  Craig W. Reynolds.  Flocks, herds, and schools: a distributed behavioral model , 1987, SIGGRAPH.

[21]  Aboul Ella Hassanien,et al.  Nature Inspired Optimization Algorithms for CT Liver Segmentation , 2016 .

[22]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[23]  Aboul Ella Hassanien,et al.  Two biometric approaches for cattle identification based on features and classifiers fusion , 2015 .

[24]  Leandro dos Santos Coelho,et al.  Gaussian quantum-behaved particle swarm optimization approaches for constrained engineering design problems , 2010, Expert Syst. Appl..

[25]  René Thomsen,et al.  A comparative study of differential evolution, particle swarm optimization, and evolutionary algorithms on numerical benchmark problems , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[26]  Gerald Schaefer,et al.  Ear Recognition Using Block-Based Principal Component Analysis and Decision Fusion , 2015, PReMI.

[27]  M. A. Abido.  Optimal des'ign of Power System Stabilizers Using Particle Swarm Opt'imization , 2002, IEEE Power Engineering Review.

[28]  Kashif Ishaque,et al.  A Deterministic Particle Swarm Optimization Maximum Power Point Tracker for Photovoltaic System Under Partial Shading Condition , 2013, IEEE Transactions on Industrial Electronics.

[29]  Nilanjan Dey,et al.  Plants Identification Using Feature Fusion Technique and Bagging Classifier , 2015, AISI.

[30]  Mohammad Teshnehlab,et al.  Parameter optimization of improved fuzzy c-means clustering algorithm for brain MR image segmentation , 2010, Eng. Appl. Artif. Intell..

[31]  Aboul Ella Hassanien,et al.  Moth-flame optimization for training Multi-Layer Perceptrons , 2015, 2015 11th International Computer Engineering Conference (ICENCO).

[32]  Alaa Tharwat,et al.  Linear vs. quadratic discriminant analysis classifier: a tutorial , 2016, Int. J. Appl. Pattern Recognit..

[33]  Masafumi Miyatake,et al.  Maximum Power Point Tracking of Multiple Photovoltaic Arrays: A PSO Approach , 2011, IEEE Transactions on Aerospace and Electronic Systems.

[34]  Aboul Ella Hassanien,et al.  A Predictive Model for Toxicity Effects Assessment of Biotransformed Hepatic Drugs Using Iterative Sampling Method , 2016, Scientific Reports.

[35]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[36]  Narayana Prasad Padhy,et al.  Optimal location and controller design of STATCOM for power system stability improvement using PSO , 2008, J. Frankl. Inst..

[37]  Ganesh K. Venayagamoorthy,et al.  Particle Swarm Optimization in Wireless-Sensor Networks: A Brief Survey , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[38]  Yoshikazu Fukuyama,et al.  A particle swarm optimization for reactive power and voltage control considering voltage security assessment , 2000 .

[39]  Abdulhamit Subasi,et al.  Classification of EMG signals using PSO optimized SVM for diagnosis of neuromuscular disorders , 2013, Comput. Biol. Medicine.