Quantum-Inspired Optimization of Echo State Networks Applied to System Identification

Quantum-Inspired Evolutionary Algorithms (QIEA) represent an efficient alternative to the traditional genetic algorithms, being capable of finding good solutions with smaller populations. Echo State Networks (ESNs) are a simple and efficient implementation of the Reservoir Computing framework. The use of this kind of networks in system identification is advantageous due to its intrinsic dynamic behavior and fast training procedure. However, ESNs have global parameters that should be tuned in order to improve their performance in a determined task. Besides, the random generation of the reservoir weights of these networks may not be ideal in terms of performance. Thus, this work presents a method that automatically defines an ESN for system identification problems by using a real coded QIEA (QIEA-R) in a two-phase optimization procedure. The QIEA-R firstly searches for the best global parameters of an ESN; then, on a second stage, optimizes some of its reservoir weights. In two benchmark problems for system identification, the proposed method overcame the performance of a randomly generated ESN with the same global parameters and has presented comparable and, in most cases, better accuracy results in comparison to some methods which were applied to the same datasets.

[1]  S. Billings.  Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Temporal Domains , 2013 .

[2]  Benjamin Schrauwen,et al.  An overview of reservoir computing: theory, applications and implementations , 2007, ESANN.

[3]  UMA NOVA ABORDAGEM VECTOR FITTING PARA IDENTIFICAÇÃO DE SISTEMAS COM DADOS NO DOMÍNIO DO TEMPO , 2015 .

[4]  Johan Schoukens,et al.  Three free data sets for development and benchmarking in nonlinear system identification , 2013, 2013 European Control Conference (ECC).

[5]  Xiao-Feng Lin,et al.  Modelling of the vertical raw cement mill grinding process based on the echo state network , 2016, 2016 12th World Congress on Intelligent Control and Automation (WCICA).

[6]  Nikolaus Hansen,et al.  Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[7]  Jong-Hwan Kim,et al.  Quantum-inspired evolutionary algorithm for a class of combinatorial optimization , 2002, IEEE Trans. Evol. Comput..

[8]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[9]  John G. Harris,et al.  Automatic speech recognition using a predictive echo state network classifier , 2007, Neural Networks.

[10]  Dwi Pebrianti,et al.  Simultaneous Computation of Model Order and Parameter Estimation for System Identification Based on Gravitational Search Algorithm , 2015, 2015 6th International Conference on Intelligent Systems, Modelling and Simulation.

[11]  Marley M. B. R. Vellasco,et al.  A new model for credit approval problems: A quantum-inspired neuro-evolutionary algorithm with binary-real representation , 2009, 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC).

[12]  O. Nelles.  Nonlinear System Identification: From Classical Approaches to Neural Networks and Fuzzy Models , 2000 .

[13]  Michael Defoin-Platel,et al.  Quantum-Inspired Evolutionary Algorithm: A Multimodel EDA , 2009, IEEE Transactions on Evolutionary Computation.

[14]  Adel M. Alimi,et al.  PSO-based analysis of Echo State Network parameters for time series forecasting , 2017, Appl. Soft Comput..

[15]  Teresa Bernarda Ludermir,et al.  Genetic algorithm for reservoir computing optimization , 2009, 2009 International Joint Conference on Neural Networks.

[16]  Lee Spector,et al.  Inference of compact nonlinear dynamic models by epigenetic local search , 2016, Eng. Appl. Artif. Intell..

[17]  Mantas Lukosevicius,et al.  A Practical Guide to Applying Echo State Networks , 2012, Neural Networks: Tricks of the Trade.

[18]  Fujimoto Kenji,et al.  System identification of nonlinear state-space models based on variational Bayes: how to cope with multimodality of variational posterior distributions , 2017 .

[19]  Teresa Bernarda Ludermir,et al.  An approach to reservoir computing design and training , 2013, Expert Syst. Appl..

[20]  M. Pacheco,et al.  Quantum-Inspired Evolutionary Algorithm for Numerical Optimization , 2006 .

[21]  Thomas B. Schön,et al.  System identification of nonlinear state-space models , 2011, Autom..

[22]  T. van der Zant,et al.  Identification of motion with echo state network , 2004, Oceans '04 MTS/IEEE Techno-Ocean '04 (IEEE Cat. No.04CH37600).

[23]  Amer Draa,et al.  A new real-coded quantum-inspired evolutionary algorithm for continuous optimization , 2017, Appl. Soft Comput..

[24]  Herbert Jaeger,et al.  A tutorial on training recurrent neural networks , covering BPPT , RTRL , EKF and the " echo state network " approach - Semantic Scholar , 2005 .

[25]  Vladimir Ceperic,et al.  Echo State Networks for Black-Box Modeling of Integrated Circuits , 2016, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[26]  Enrique Alba,et al.  An experimental analysis of the Echo State Network initialization using the Particle Swarm Optimization , 2014, 2014 Sixth World Congress on Nature and Biologically Inspired Computing (NaBIC 2014).

[27]  Roberto Schirru,et al.  Closed Loop Identification of Nuclear Steam Generator Water Level Using ESN Network Tuned by Genetic Algorithm , 2015, EANN.