Dynamic mentoring and self-regulation based particle swarm optimization algorithm for solving complex real-world optimization problems

Abstract In this paper, a dynamic mentoring scheme along with a self-regulation scheme have been incorporated in the standard Particle Swarm Optimization (PSO) algorithm to empower the searching particles with human-like characteristics. The algorithm is referred to as a Dynamic Mentoring and Self-Regulation based Particle Swarm Optimization (DMeSR-PSO) algorithm. Based on their experiences, the particles are divided into three groups, viz., the mentor group, the mentee group and the independent learner group where the number of particles in each group is dynamically changing in every iteration. In human learning psychology, mentoring is regarded as a powerful and effective learning process and independent learners are the ones who do not need mentoring and are capable of performing self-regulation of their own knowledge. Therefore, the particles in each of the above three groups have different learning strategies for their velocity updates where the mentors are equipped with a strong self-belief based search, the mentees are taking guidance from the mentors and the independent learners employ self-perception strategy. The DMeSR-PSO algorithm has been extensively evaluated using the simple unimodal and multimodal benchmark functions from CEC2005, more complex shifted and rotated benchmark functions from CEC2013 and also based on eight real-world problems from CEC2011. The results have been compared with six state-of-the-art PSO variants and five meta-heuristic algorithms for the CEC2005 problems. Further, a comparative analysis on CEC2013 benchmark functions with different PSO variants has also been presented. Finally, DMeSR-PSO’s performance on the real-world problems is compared with the top two algorithms from the CEC2011 competition. The results indicate that the proposed learning strategies help DMeSR-PSO to achieve faster convergence and provide better solutions in most of the problems with a 95% confidence level, yielding an effective optimization algorithm for real-world applications.

[1]  Konstantinos E. Parsopoulos,et al.  UPSO: A Unified Particle Swarm Optimization Scheme , 2019, International Conference of Computational Methods in Sciences and Engineering 2004 (ICCMSE 2004).

[2]  Chukwudi Anyakoha,et al.  A review of particle swarm optimization. Part II: hybridisation, combinatorial, multicriteria and constrained optimization, and indicative applications , 2008, Natural Computing.

[3]  C. Floudas.  Handbook of Test Problems in Local and Global Optimization , 1999 .

[4]  L. A. Daloz.  Mentor : Guiding the Journey of Adult Learners , 1999 .

[5]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[6]  Andries Petrus Engelbrecht,et al.  A Cooperative approach to particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[7]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[8]  Jing J. Liang,et al.  Dynamic multi-swarm particle swarm optimizer , 2005, Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005..

[9]  Sancho Salcedo-Sanz,et al.  A hybrid evolutionary programming algorithm for spread spectrum radar polyphase codes design , 2007, GECCO '07.

[10]  Jirí Benes,et al.  On neural networks , 1990, Kybernetika.

[11]  Michael G. Epitropakis,et al.  Evolving cognitive and social experience in Particle Swarm Optimization through Differential Evolution: A hybrid approach , 2012, Inf. Sci..

[12]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[13]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[14]  Claudia C. Mincemoyer,et al.  Establishing Effective Mentoring Relationships for Individual and Organizational Success. , 1998 .

[15]  Sundaram Suresh,et al.  Human cognition inspired particle swarm optimization algorithm , 2014, 2014 IEEE Ninth International Conference on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP).

[16]  Jun Zhang,et al.  Competitive and cooperative particle swarm optimization with information sharing mechanism for global optimization problems , 2015, Inf. Sci..

[17]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[18]  Jeffery D. Weir,et al.  AHPS2: An optimizer using adaptive heterogeneous particle swarms , 2014, Inf. Sci..

[19]  Andries P. Engelbrecht,et al.  Computational Intelligence: An Introduction , 2002 .

[20]  Yaochu Jin,et al.  A social learning particle swarm optimization algorithm for scalable optimization , 2015, Inf. Sci..

[21]  A. Rezaee Jordehi,et al.  Parameter selection in particle swarm optimisation: a survey , 2013, J. Exp. Theor. Artif. Intell..

[22]  James Kennedy,et al.  Bare bones particle swarms , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[23]  Ruben Romero,et al.  Transmission network expansion planning with security constraints , 2005 .

[24]  Ziyang Liu,et al.  A multi-swarm cooperative multistage perturbation guiding particle swarm optimizer , 2014, Appl. Soft Comput..

[25]  Dervis Karaboga,et al.  A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm , 2007, J. Glob. Optim..

[26]  T. O. Nelson.  Metamemory: A Theoretical Framework and New Findings , 1990 .

[27]  Narasimhan Sundararajan,et al.  Self regulating particle swarm optimization algorithm , 2015, Inf. Sci..

[28]  Narasimhan Sundararajan,et al.  Human meta-cognition inspired collaborative search algorithm for optimization , 2014, 2014 International Conference on Multisensor Fusion and Information Integration for Intelligent Systems (MFI).

[29]  汤可宗,et al.  Multi-strategy adaptive particle swarm optimization for numerical optimization , 2015 .

[30]  José Neves,et al.  The fully informed particle swarm: simpler, maybe better , 2004, IEEE Transactions on Evolutionary Computation.

[31]  Swagatam Das,et al.  Behavioral analysis of the leader particle during stagnation in a particle swarm optimization algorithm , 2014, Inf. Sci..

[32]  Nor Ashidi Mat Isa,et al.  An adaptive two-layer particle swarm optimization with elitist learning strategy , 2014, Inf. Sci..

[33]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[34]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[35]  Lifang Xu,et al.  A power spectrum optimization algorithm inspired by magnetotactic bacteria , 2014, Neural Computing and Applications.

[36]  Guangqing Bao,et al.  Particle swarm optimization algorithm with asymmetric time varying acceleration coefficients , 2009, 2009 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[37]  Jing J. Liang,et al.  Comprehensive learning particle swarm optimizer for global optimization of multimodal functions , 2006, IEEE Transactions on Evolutionary Computation.

[38]  Nor Ashidi Mat Isa,et al.  Particle swarm optimization with adaptive time-varying topology connectivity , 2014, Appl. Soft Comput..

[39]  Ahmad Bagheri,et al.  HEPSO: High exploration particle swarm optimization , 2014, Inf. Sci..

[40]  Alan S. Perelson,et al.  The immune system, adaptation, and machine learning , 1986 .

[41]  Narasimhan Sundararajan,et al.  Mentoring based particle swarm optimization algorithm for faster convergence , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[42]  Xiang Yu,et al.  Enhanced comprehensive learning particle swarm optimization , 2014, Appl. Math. Comput..

[43]  Changhe Li,et al.  Particle swarm optimisation with simple and efficient neighbourhood search strategies , 2011 .

[44]  Yang Xianfeng,et al.  Dynamic Adjustment Strategies of Inertia Weight in Particle Swarm Optimization Algorithm , 2014 .

[45]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[46]  Harun Uğuz,et al.  A novel particle swarm optimization algorithm with Levy flight , 2014, Appl. Soft Comput..

[47]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[48]  Zhang Dingxue,et al.  A Modified Particle Swarm Optimization with an Adaptive Acceleration Coefficients , 2009, 2009 Asia-Pacific Conference on Information Processing.

[49]  J. Kennedy,et al.  Population structure and particle swarm performance , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[50]  Liyan Zhang,et al.  Empirical study of particle swarm optimizer with an increasing inertia weight , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[51]  Rafael Bello,et al.  Particle Swarm Optimization with Random Sampling in Variable Neighbourhoods for Solving Global Minimization Problems , 2012, ANTS.

[52]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[53]  Dan Simon,et al.  Biogeography-Based Optimization , 2022 .

[54]  Ponnuthurai N. Suganthan,et al.  A Distance-Based Locally Informed Particle Swarm Model for Multimodal Optimization , 2013, IEEE Transactions on Evolutionary Computation.

[55]  H. Law.  The Psychology of Coaching, Mentoring and Learning , 2007 .

[56]  Wenhua Han,et al.  Comparison study of several kinds of inertia weights for PSO , 2010, 2010 IEEE International Conference on Progress in Informatics and Computing.

[57]  Changhe Li,et al.  A Self-Learning Particle Swarm Optimizer for Global Optimization Problems , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[58]  Mohammad Reza Meybodi,et al.  novel multi-swarm algorithm for optimization in dynamic environments based n particle swarm optimization , 2013 .

[59]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[60]  Ruhul A. Sarker,et al.  Differential evolution with multiple strategies for solving CEC2011 real-world numerical optimization problems , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[61]  P. N. Suganthan,et al.  Problem Definitions and Evaluation Criteria for CEC 2011 Competition on Testing Evolutionary Algorithms on Real World Optimization Problems , 2011 .

[62]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[63]  Ankit Chaudhary,et al.  A comparative review of approaches to prevent premature convergence in GA , 2014, Appl. Soft Comput..

[64]  James Kennedy,et al.  Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[65]  Feng Zou,et al.  Teaching-learning-based optimization with dynamic group strategy for global optimization , 2014, Inf. Sci..

[66]  Yuhui Shi,et al.  Brain Storm Optimization Algorithm , 2011, ICSI.

[67]  Z. Beheshti.  A review of population-based meta-heuristic algorithm , 2013, SOCO 2013.

[68]  Ruhul A. Sarker,et al.  GA with a new multi-parent crossover for solving IEEE-CEC2011 competition problems , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[69]  Xu Chen,et al.  A Novel Particle Swarm Optimization Based on the Self-Adaptation Strategy of Acceleration Coefficients , 2009, 2009 International Conference on Computational Intelligence and Security.

[70]  Jeng-Shyang Pan,et al.  A new fitness estimation strategy for particle swarm optimization , 2013, Inf. Sci..

[71]  Pascal Bouvry,et al.  Particle swarm optimization: Hybridization perspectives and experimental illustrations , 2011, Appl. Math. Comput..

[72]  Azah Mohamed,et al.  A Survey of the State of the Art in Particle Swarm Optimization , 2012 .

[73]  Riccardo Poli,et al.  Analysis of the publications on the applications of particle swarm optimisation , 2008 .

[74]  Kalyan Veeramachaneni,et al.  Fitness-distance-ratio based particle swarm optimization , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[75]  R. Venkata Rao,et al.  Mechanical Design Optimization Using Advanced Optimization Techniques , 2012 .