Experienced Grey Wolf Optimizer through Reinforcement Learning and Neural Networks

In this paper, a variant of Grey Wolf Optimizer (GWO) that uses reinforcement learning principles combined with neural networks to enhance the performance is proposed. The aim is to overcome, by reinforced learning, the common challenges of setting the right parameters for the algorithm. In GWO, a single parameter is used to control the exploration/exploitation rate which influences the performance of the algorithm. Rather than using a global way to change this parameter for all the agents, we use reinforcement learning to set it on an individual basis. The adaptation of the exploration rate for each agent depends on the agent’s own experience and the current terrain of the search space. In order to achieve this, an experience repository is built based on the neural network to map a set of agents’ states to a set of corresponding actions that specifically influence the exploration rate. The experience repository is updated by all the search agents to reflect experience and to enhance the future actions continuously. The resulted algorithm is called Experienced Grey Wolf Optimizer (EGWO) and its performance is assessed on solving feature selection problems and on finding optimal weights for neural networks algorithm. We use a set of performance indicators to evaluate the efficiency of the method. Results over various datasets demonstrate an advance of the EGWO over the original GWO and other meta-heuristics such as genetic algorithms and particle swarm optimization.

[1]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[2]  E. L. Lehmann,et al.  Theory of point estimation , 1950 .

[3]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[4]  J. Rice Mathematical Statistics and Data Analysis , 1988 .

[5]  Xin Yao,et al.  A review of evolutionary artificial neural networks , 1993, Int. J. Intell. Syst..

[6]  A. E. Eiben,et al.  Genetic algorithms with multi-parent recombination , 1994, PPSN.

[7]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[8]  Yu Li,et al.  Particle swarm optimisation for evolving artificial neural network , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[9]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[10]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[11]  Michael R. Lyu,et al.  A hybrid particle swarm optimization-back-propagation algorithm for feedforward neural network training , 2007, Appl. Math. Comput..

[12]  Li-Yeh Chuang,et al.  Improved binary PSO for feature selection using gene expression data , 2008, Comput. Biol. Chem..

[13]  Wenyin Gong,et al.  An Improved Self-adaptive Control Parameter of Differential Evolution for Global Optimization , 2009 .

[14]  Driss Aboutajdine,et al.  A two-stage gene selection scheme utilizing MRMR filter and GA wrapper , 2011, Knowledge and Information Systems.

[15]  Li-Yeh Chuang,et al.  Improved binary particle swarm optimization using catfish effect for feature selection , 2011, Expert Syst. Appl..

[16]  H. Seo,et al.  Neural basis of reinforcement learning and decision making. , 2012, Annual review of neuroscience.

[17]  Xin-She Yang,et al.  Swarm Intelligence and Bio-Inspired Computation , 2013 .

[18]  Andrew Lewis,et al.  Grey Wolf Optimizer , 2014, Adv. Eng. Softw..

[19]  Seyed Mohammad Mirjalili How effective is the Grey Wolf optimizer in training multi-layer perceptrons , 2014, Applied Intelligence.

[20]  Mengjie Zhang,et al.  Particle swarm optimisation for feature selection in classification: Novel initialisation and updating mechanisms , 2014, Appl. Soft Comput..

[21]  Surafel Luleseged Tilahun,et al.  Prey-Predator Algorithm: A New Metaheuristic Algorithm for Optimization Problems , 2015, Int. J. Inf. Technol. Decis. Mak..

[22]  Aboul Ella Hassanien,et al.  Binary grey wolf optimization approaches for feature selection , 2016, Neurocomputing.

[23]  Leandro dos Santos Coelho,et al.  Multi-objective grey wolf optimizer: A novel algorithm for multi-criterion optimization , 2016, Expert Syst. Appl..

[24]  Wei Qiao,et al.  An Adaptive Network-Based Reinforcement Learning Method for MPPT Control of PMSG Wind Energy Conversion Systems , 2016, IEEE Transactions on Power Electronics.

[25]  Urvinder Singh,et al.  Modified Grey Wolf Optimizer for Global Engineering Optimization , 2016, Appl. Comput. Intell. Soft Comput..

[26]  Huaguang Zhang,et al.  Data-Driven Optimal Consensus Control for Discrete-Time Multi-Agent Systems With Unknown Dynamics Using Reinforcement Learning Method , 2017, IEEE Transactions on Industrial Electronics.

[27]  Huaguang Zhang,et al.  Adaptive Fault-Tolerant Tracking Control for MIMO Discrete-Time Systems via Reinforcement Learning Algorithm With Less Learning Parameters , 2017, IEEE Transactions on Automation Science and Engineering.

[28]  Haibo He,et al.  Manifold-Based Reinforcement Learning via Locally Linear Reconstruction , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Youyong Kong,et al.  Deep Direct Reinforcement Learning for Financial Signal Representation and Trading , 2017, IEEE Transactions on Neural Networks and Learning Systems.