Applications of Metaheuristics in Reservoir Computing Techniques: A Review

Reservoir computing approaches have been around for almost two decades. They were developed to solve the difficult gradient-descent training of recurrent neural networks. However, in reservoir computing, the choice of parameters and architecture often leads to an optimization challenge. Most early applications have used trial and error with expert knowledge to select the right value for parameter(s)/architecture. This approach is usually cumbersome and difficult considering the large search space. Metaheuristics have been known to perform well in solving such kinds of problems. This review discusses areas where metaheuristics are used in the echo state network—a pioneer in the reservoir computing field. In addition, trends and research gaps are also discussed.

[1]  Zuren Feng,et al.  Stable training method for echo state networks with output feedbacks , 2010, 2010 International Conference on Networking, Sensing and Control (ICNSC).

[2]  Ming Fu,et al.  A Multi Time Scale Wind Power Forecasting Model of a Chaotic Echo State Network Based on a Hybrid Algorithm of Particle Swarm Optimization and Tabu Search , 2015 .

[3]  Herbert Jaeger,et al.  Overview of Reservoir Recipes , 2007 .

[4]  Jan-Yan Lin,et al.  Extended modeling procedure based on the projected sample for forecasting short-term electricity consumption , 2016, Adv. Eng. Informatics.

[5]  Jürgen Schmidhuber,et al.  Learning to forget: continual prediction with LSTM , 1999 .

[6]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[7]  Zuren Feng,et al.  Stable trajectory generator-echo state network trained by particle swarm optimization , 2009, 2009 IEEE International Symposium on Computational Intelligence in Robotics and Automation - (CIRA).

[8]  Alberto J. Alvares,et al.  Prognostic of RUL based on Echo State Network Optimized by Artificial Bee Colony , 2020 .

[9]  Sadiq M. Sait,et al.  Iterative computer algorithms with applications in engineering - solving combinatorial optimization problems , 2000 .

[10]  Li Wei,et al.  Electrical Load Forecasting Using Echo State Network and Optimizing by PSO Algorithm , 2017, 2017 10th International Conference on Intelligent Computation Technology and Automation (ICICTA).

[11]  Petros Koumoutsakos,et al.  Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES) , 2003, Evolutionary Computation.

[12]  Enrico Zio,et al.  Echo State Network for the Remaining Useful Life Prediction of a Turbofan Engine , 2016, PHM Society European Conference.

[13]  Rosdiazli Ibrahim,et al.  Cuckoo Search Based Optimization of Echo State Network for Time Series Prediction , 2018, 2018 International Conference on Intelligent and Advanced System (ICIAS).

[14]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[15]  Gillian Dobbie,et al.  Research on particle swarm optimization based clustering: A systematic review of literature and techniques , 2014, Swarm Evol. Comput..

[16]  Yu Peng,et al.  A modified echo state network based remaining useful life estimation approach , 2012, 2012 IEEE Conference on Prognostics and Health Management.

[17]  Teresa Bernarda Ludermir,et al.  Using Reservoir Computing for Forecasting Time Series: Brazilian Case Study , 2008, 2008 Eighth International Conference on Hybrid Intelligent Systems.

[18]  Mantas Lukosevicius,et al.  A Practical Guide to Applying Echo State Networks , 2012, Neural Networks: Tricks of the Trade.

[19]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[20]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[21]  Teresa Bernarda Ludermir,et al.  Comparing evolutionary methods for reservoir computing pre-training , 2011, The 2011 International Joint Conference on Neural Networks.

[22]  André Gustavo Maletzke,et al.  Exploring shapelet transformation for time series classification in decision trees , 2016, Knowl. Based Syst..

[23]  Teresa Bernarda Ludermir,et al.  An approach to reservoir computing design and training , 2013, Expert Syst. Appl..

[24]  Kenneth Morgan,et al.  Modified cuckoo search: A new gradient free optimisation algorithm , 2011 .

[25]  Claudio Gallicchio,et al.  Architectural and Markovian factors of echo state networks , 2011, Neural Networks.

[26]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[27]  Peter Tiño,et al.  Minimum Complexity Echo State Network , 2011, IEEE Transactions on Neural Networks.

[28]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[29]  Emile H. L. Aarts,et al.  Performance of the simulated annealing algorithm , 1987 .

[30]  Lin Wang,et al.  Effective electricity energy consumption forecasting using echo state network improved by differential evolution algorithm , 2018 .

[31]  T. van der Zant,et al.  Identification of motion with echo state network , 2004, Oceans '04 MTS/IEEE Techno-Ocean '04 (IEEE Cat. No.04CH37600).

[32]  Guang-Bin Huang,et al.  An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels , 2014, Cognitive Computation.

[33]  Adel M. Alimi,et al.  A Hybrid Approach Based on Particle Swarm Optimization for Echo State Network Initialization , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[34]  Zong Woo Geem,et al.  A New Heuristic Optimization Algorithm: Harmony Search , 2001, Simul..

[35]  Dan Simon,et al.  Biogeography-Based Optimization , 2022 .

[36]  Kai Liu,et al.  Spectrum Prediction Based on Echo State Network and Its Improved Form , 2013, 2013 5th International Conference on Intelligent Human-Machine Systems and Cybernetics.

[37]  Teresa Bernarda Ludermir,et al.  Genetic algorithm for reservoir computing optimization , 2009, 2009 International Joint Conference on Neural Networks.

[38]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[39]  S. Arora,et al.  A conceptual comparison of firefly algorithm, bat algorithm and cuckoo search , 2013, 2013 International Conference on Control, Computing, Communication and Materials (ICCCCM).

[40]  Thong Ngee Goh,et al.  A comparative study of neural network and Box-Jenkins ARIMA modeling in time series prediction , 2002 .

[41]  Md Abu Shahab Mollah,et al.  Sensitivity learning oriented nonmonotonic multi reservoir echo state network for short-term load forecasting , 2013, 2013 International Conference on Informatics, Electronics and Vision (ICIEV).

[42]  Ibrahim Eksin,et al.  A new optimization method: Big Bang-Big Crunch , 2006, Adv. Eng. Softw..

[43]  Chao-Ming Huang,et al.  A particle swarm optimization to identifying the ARMAX model for short-term load forecasting , 2005 .

[44]  Min Han,et al.  Hierarchical neural networks for multivariate time series prediction , 2016, 2016 35th Chinese Control Conference (CCC).

[45]  Jürgen Schmidhuber,et al.  Modeling systems with internal state using evolino , 2005, GECCO '05.

[46]  Jianzhou Wang,et al.  Application of residual modification approach in seasonal ARIMA for electricity demand forecasting: A case study of China , 2012 .

[47]  Simon Haykin,et al.  Decoupled echo state networks with lateral inhibition , 2007, Neural Networks.

[48]  Antonello Rizzi,et al.  Short-Term Electric Load Forecasting Using Echo State Networks and PCA Decomposition , 2015, IEEE Access.

[49]  Adel M. Alimi,et al.  Designing Beta Basis Function Neural Network for optimization using Artificial Bee Colony (ABC) , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[50]  Amir Hussain,et al.  Multilayered Echo State Machine: A Novel Architecture and Algorithm , 2017, IEEE Transactions on Cybernetics.

[51]  Prajakta S. Kalekar.  Time series Forecasting using Holt-Winters Exponential Smoothing , 2004 .

[52]  Xiaohua Wang,et al.  Echo State Networks With Orthogonal Pigeon-Inspired Optimization for Image Restoration , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[53]  Enrico Zio,et al.  Ensemble of optimized echo state networks for remaining useful life prediction , 2017, Neurocomputing.

[54]  Andreas Zell,et al.  Optimizing recurrent reservoirs with neuro-evolution , 2016, Neurocomputing.

[55]  Adel M. Alimi,et al.  Single- and multi-objective particle swarm optimization of reservoir structure in Echo State Network , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[56]  Kathryn A. Dowsland,et al.  Simulated Annealing , 1989, Encyclopedia of GIS.

[57]  Dervis Karaboga,et al.  Artificial Bee Colony (ABC) Optimization Algorithm for Solving Constrained Optimization Problems , 2007, IFSA.

[58]  Robert Jenssen,et al.  Training Echo State Networks with Regularization Through Dimensionality Reduction , 2016, Cognitive Computation.

[59]  Adel M. Alimi,et al.  The Modified Differential Evolution and the RBF (MDE-RBF) Neural Network for Time Series Prediction , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[60]  Deming Liu,et al.  The Application of modified ESN in chaotic time series prediction , 2013, 2013 25th Chinese Control and Decision Conference (CCDC).

[61]  Wei Wang,et al.  Map-reduce framework-based non-iterative granular echo state network for prediction intervals construction , 2017, Neurocomputing.

[62]  Garrison W. Cottrell,et al.  Deep-ESN: A Multiple Projection-encoding Hierarchical Reservoir Computing Framework , 2017, ArXiv.

[63]  Wolfgang Maass,et al.  Liquid State Machines: Motivation, Theory, and Applications , 2010 .

[64]  Benjamin Schrauwen,et al.  The Introduction of Time-Scales in Reservoir Computing, Applied to Isolated Digits Recognition , 2007, ICANN.

[65]  Junfei Qiao,et al.  A novel echo state network design method based on differential evolution algorithm , 2017, 2017 36th Chinese Control Conference (CCC).

[66]  Dario Floreano,et al.  Neuroevolution with Analog Genetic Encoding , 2006, PPSN.

[67]  Wen Gao,et al.  Group-Based Sparse Representation for Image Restoration , 2014, IEEE Transactions on Image Processing.

[68]  K. Doya,et al.  Bifurcations in the learning of recurrent neural networks , 1992, [Proceedings] 1992 IEEE International Symposium on Circuits and Systems.

[69]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[70]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[71]  Marc Schoenauer,et al.  Supervised and Evolutionary Learning of Echo State Networks , 2008, PPSN.

[72]  Jun Zhang,et al.  Adaptive Particle Swarm Optimization , 2008, ANTS Conference.

[73]  Kun Wang,et al.  The Application of Liquid State Machines in Robot Path Planning , 2009, J. Comput..

[74]  Abdolreza Hatamlou,et al.  Black hole: A new heuristic optimization approach for data clustering , 2013, Inf. Sci..

[75]  Dongming Xu,et al.  Direct adaptive control: an echo state network and genetic algorithm approach , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[76]  Marc Schoenauer,et al.  Unsupervised Learning of Echo State Networks: A Case Study in Artificial Embryogeny , 2007, Artificial Evolution.

[77]  J.J. Steil,et al.  Backpropagation-decorrelation: online recurrent learning with O(N) complexity , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[78]  B. Yegnanarayana,et al.  Artificial Neural Networks , 2004 .

[79]  Shang-Jeng Tsai,et al.  Efficient Population Utilization Strategy for Particle Swarm Optimizer , 2009, IEEE Trans. Syst. Man Cybern. Part B.

[80]  Lin Lin,et al.  Genetic algorithm optimized double-reservoir echo state network for multi-regime time series prediction , 2017, Neurocomputing.

[81]  Benjamin Schrauwen,et al.  Reservoir Computing Trends , 2012, KI - Künstliche Intelligenz.

[82]  Xuefeng Yan,et al.  Optimizing the echo state network with a binary particle swarm optimization algorithm , 2015, Knowl. Based Syst..

[83]  Yongji Wang,et al.  Modeling of Cortical Signals Using Optimized Echo State Networks with Leaky Integrator Neurons , 2009, ICONIP.

[84]  Min Han,et al.  Support Vector Echo-State Machine for Chaotic Time-Series Prediction , 2007, IEEE Transactions on Neural Networks.

[85]  Jaakko Astola,et al.  Directional varying scale approximations for anisotropic signal processing , 2004, 2004 12th European Signal Processing Conference.

[86]  Junfei Qiao,et al.  Growing Echo-State Network With Multiple Subreservoirs , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[87]  Tim Oates,et al.  Encoding Time Series as Images for Visual Inspection and Classification Using Tiled Convolutional Neural Networks , 2014 .

[88]  Ponnuthurai N. Suganthan,et al.  Recent advances in differential evolution - An updated survey , 2016, Swarm Evol. Comput..

[89]  Christine M. Anderson-Cook,et al.  Practical Genetic Algorithms , 2005 .

[90]  Andrew Lewis,et al.  Grey Wolf Optimizer , 2014, Adv. Eng. Softw..

[91]  Xin-She Yang,et al.  A New Metaheuristic Bat-Inspired Algorithm , 2010, NICSO.

[92]  Enrique Alba,et al.  An experimental analysis of the Echo State Network initialization using the Particle Swarm Optimization , 2014, 2014 Sixth World Congress on Nature and Biologically Inspired Computing (NaBIC 2014).

[93]  Kamran Shafi,et al.  Learning Structures of Conceptual Models from Observed Dynamics Using Evolutionary Echo State Networks , 2018, J. Artif. Intell. Soft Comput. Res..

[94]  Qian Feng,et al.  θ-PSO: a new strategy of particle swarm optimization , 2008 .

[95]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[96]  Ali Deihimi,et al.  Application of echo state networks in short-term electric load forecasting , 2012 .

[97]  Zuren Feng,et al.  Stable Training Method for Echo State Networks Running in Closed-Loop Based on Particle Swarm Optimization Algorithm , 2009, ICONIP.

[98]  Farshid Keynia,et al.  Short-term load forecasting of power systems by combination of wavelet transform and neuro-evolutionary algorithm , 2009 .

[99]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[100]  Adel M. Alimi,et al.  Hierarchical Bi-level Multi-Objective Evolution of Single- and Multi-layer Echo State Network Autoencoders for Data Representations , 2018, ArXiv.

[101]  Anderson Tenório Sergio,et al.  Reservoir Computing optimization with a hybrid method , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[102]  Michèle Sebag,et al.  Memory-enhanced Evolutionary Robotics: The Echo State Network Approach , 2009, 2009 IEEE Congress on Evolutionary Computation.

[103]  Jie-Sheng Wang,et al.  KPCA-ESN Soft-Sensor Model of Polymerization Process Optimized by Biogeography-Based Optimization Algorithm , 2015 .

[104]  J. Michael Herrmann,et al.  A Review of No Free Lunch Theorems, and Their Implications for Metaheuristic Optimisation , 2018 .

[105]  Risto Miikkulainen,et al.  Efficient Reinforcement Learning Through Evolving Neural Network Topologies , 2002, GECCO.

[106]  Leandro dos Santos Coelho,et al.  Earthworm optimisation algorithm: a bio-inspired metaheuristic algorithm for global optimisation problems , 2018, Int. J. Bio Inspired Comput..

[107]  Teresa Bernarda Ludermir,et al.  PSO for Reservoir Computing Optimization , 2012, ICANN.

[108]  Kehe Wu,et al.  Algorithm and Implementation of Distributed ESN Using Spark Framework and Parallel PSO , 2017 .

[109]  Darrell Whitley,et al.  A genetic algorithm tutorial , 1994, Statistics and Computing.

[110]  Adel M. Alimi,et al.  PSO-based analysis of Echo State Network parameters for time series forecasting , 2017, Appl. Soft Comput..

[111]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[112]  Koffka Khan,et al.  A Comparison of BA, GA, PSO, BP and LM for Training Feed forward Neural Networks in e-Learning Context , 2012 .

[113]  Ping-Feng Pai,et al.  Forecasting regional electricity load based on recurrent support vector machines with genetic algorithms , 2005 .

[114]  Christian Blum,et al.  Hybrid Metaheuristics: An Introduction , 2008, Hybrid Metaheuristics.

[115]  Moncef Tagina,et al.  Echo State Network and Particle Swarm Optimization for Prognostics of a Complex System , 2017, 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA).

[116]  Stefan J. Kiebel,et al.  Re-visiting the echo state property , 2012, Neural Networks.

[117]  Dervis Karaboga,et al.  A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm , 2007, J. Glob. Optim..

[118]  Richard G. Baraniuk,et al.  ForWaRD: Fourier-wavelet regularized deconvolution for ill-conditioned systems , 2004, IEEE Transactions on Signal Processing.

[119]  Herbert Jaeger,et al.  Optimization and applications of echo state networks with leaky- integrator neurons , 2007, Neural Networks.

[120]  Abolfazl Rahmani,et al.  Application of echo state networks for estimating voltage harmonic waveforms in power systems considering a photovoltaic system , 2017 .

[121]  Enrique F. Castillo,et al.  A General Method for Local Sensitivity Analysis With Application to Regression Models and Other Optimization Problems , 2004, Technometrics.

[122]  Yourong Li,et al.  Short-term fault prediction based on support vector machines with parameter optimization by evolution strategy , 2009, Expert Syst. Appl..

[123]  Haibin Duan,et al.  Pigeon-inspired optimization: a new swarm intelligence optimizer for air robot path planning , 2014, Int. J. Intell. Comput. Cybern..

[124]  Tijn van der Zant,et al.  Finding good Echo State Networks to control an underwater robot using evolutionary computations , 2004 .

[125]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[126]  Peter Ford Dominey.  Complex sensory-motor sequence learning based on recurrent state representation and reinforcement learning , 1995, Biological Cybernetics.

[127]  L. Larger,et al.  Reservoir Computing Optimisation for PEM Fuel Cell Fault Diagnostic , 2017, 2017 IEEE Vehicle Power and Propulsion Conference (VPPC).

[128]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[129]  Benjamin Schrauwen,et al.  An overview of reservoir computing: theory, applications and implementations , 2007, ESANN.

[130]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[131]  Yongji Wang,et al.  Modeling of McKibben Pneumatic artificial muscles using optimized Echo State Networks , 2010, 2010 8th World Congress on Intelligent Control and Automation.

[132]  John J. Hopfield,et al.  Hopfield network , 2007, Scholarpedia.

[133]  Simone Scardapane,et al.  Prediction of telephone calls load using Echo State Network with exogenous variables , 2015, Neural Networks.

[134]  Qian Zhang,et al.  Research on Mid-long Term Load Forecasting Base on Wavelet Neural Network , 2010, 2010 Second International Conference on Computer Engineering and Applications.

[135]  Benjamin Schrauwen,et al.  Connectivity, Dynamics, and Memory in Reservoir Computing with Binary and Analog Neurons , 2010, Neural Computation.

[136]  A. Owen.  A robust hybrid of lasso and ridge regression , 2006 .

[137]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[138]  Cheng-jian Wei,et al.  Harnessing Non-linearity by Sigmoid-wavelet Hybrid Echo State Networks (SWHESN) , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[139]  Kun Li,et al.  Multi-step prediction for the network traffic based on echo state network optimized by quantum-behaved fruit fly optimization algorithm , 2017, 2017 29th Chinese Control And Decision Conference (CCDC).

[140]  B. Schrauwen,et al.  Reservoir computing and extreme learning machines for non-linear time-series data analysis , 2013, Neural Networks.

[141]  Lina Pan,et al.  An improved echo state network based on variational mode decomposition and bat optimization for Internet traffic forecasting , 2017, 2017 IEEE 17th International Conference on Communication Technology (ICCT).