Regularized Taylor Echo State Networks for Predictive Control of Partially Observed Systems

The existing neural networks suffer from partial observation while modeling and controlling dynamic systems. In this paper, a new linearized recurrent neural network, the Taylor expanded echo state network (TESN), is proposed for predictive control of partially observed dynamic systems. Two schemes of regularization, ridge regression and sparse regression, are imposed on TESNs to tackle the issue of ill-conditioned estimation. Furthermore, two estimators, lasso and elastic net, are investigated for sparse regression. Regularized learning is found to improve the estimation consistency of readout coefficients and, at the same time, suppress the accumulation of linearization residues in a prediction horizon. A series of experiments was carried out, and the results verified that regularized learning is contributive to TESNs in predictive control of partially observed dynamic systems.

[1]  Antonello Rizzi,et al.  Short-Term Electric Load Forecasting Using Echo State Networks and PCA Decomposition , 2015, IEEE Access.

[2]  Jun Wang,et al.  Model Predictive Control of Unknown Nonlinear Dynamical Systems Based on Recurrent Neural Networks , 2012, IEEE Transactions on Industrial Electronics.

[3]  Ning An,et al.  Speech Emotion Recognition Using Fourier Parameters , 2015, IEEE Transactions on Affective Computing.

[4]  L Fagiano,et al.  Efficient Model Predictive Control for Nonlinear Systems via Function Approximation Techniques , 2010, IEEE Transactions on Automatic Control.

[5]  S. Joe Qin,et al.  A survey of industrial model predictive control technology , 2003 .

[6]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[7]  Xuelong Li,et al.  Joint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection , 2014, IEEE Transactions on Cybernetics.

[8]  Mantas Lukosevicius,et al.  A Practical Guide to Applying Echo State Networks , 2012, Neural Networks: Tricks of the Trade.

[9]  Yi Cao,et al.  Nonlinear system identification for predictive control using continuous time recurrent neural networks and automatic differentiation , 2008 .

[10]  Jianyong Wang,et al.  Toward detection of aliases without string similarity , 2014, Inf. Sci..

[11]  Zhaohong Deng,et al.  Generalized Hidden-Mapping Ridge Regression, Knowledge-Leveraged Inductive Transfer Learning for Neural Networks, Fuzzy Systems and Kernel Methods , 2014, IEEE Transactions on Cybernetics.

[12]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[13]  Kazuki Nakada,et al.  Neural Implementation of Shape-Invariant Touch Counter Based on Euler Calculus , 2014, IEEE Access.

[14]  Satish Chand,et al.  Online linearization-based neural predictive control of air–fuel ratio in SI engines with PID feedback correction scheme , 2010, Neural Computing and Applications.

[15]  Amaury Lendasse,et al.  High-Performance Extreme Learning Machines: A Complete Toolbox for Big Data Applications , 2015, IEEE Access.

[16]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[17]  Pastora Vega,et al.  State space neural network. Properties and application , 1998, Neural Networks.

[18]  Alberto Cardoso,et al.  Affine Neural Network-Based Predictive Control Applied to a Distributed Solar Collector Field , 2014, IEEE Transactions on Control Systems Technology.

[19]  Jun Wang,et al.  Global Exponential Synchronization of Two Memristor-Based Recurrent Neural Networks With Time Delays via Static or Dynamic Coupling , 2015, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[20]  Jing Wang,et al.  Robust Face Recognition via Adaptive Sparse Representation , 2014, IEEE Transactions on Cybernetics.

[21]  P. Vega,et al.  Neural predictive control. Application to a highly non-linear system , 1999 .

[22]  Jun Wang,et al.  Chaotic Time Series Prediction Based on a Novel Robust Echo State Network , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[23]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[24]  Herbert Jaeger,et al.  Optimization and applications of echo state networks with leaky- integrator neurons , 2007, Neural Networks.

[25]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[26]  汪萌,et al.  Image Annotation By Multiple-Instance Learning With Discriminative Feature Mapping and Selection , 2014 .

[27]  Jun Wang,et al.  A Dynamic Feedforward Neural Network Based on Gaussian Particle Swarm Optimization and its Application for Predictive Control , 2011, IEEE Transactions on Neural Networks.

[28]  Vincent A Akpan,et al.  Nonlinear model identification and adaptive model predictive control using neural networks. , 2011, ISA transactions.

[29]  Jochen J. Steil,et al.  Regularization and stability in reservoir networks with output feedback , 2012, Neurocomputing.

[30]  Chi-Huang Lu,et al.  Wavelet Fuzzy Neural Networks for Identification and Predictive Control of Dynamic Systems , 2011, IEEE Transactions on Industrial Electronics.

[31]  Zheng Yan,et al.  Robust Model Predictive Control of Nonlinear Systems With Unmodeled Dynamics and Bounded Uncertainties Based on Neural Networks , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Mang I Vai,et al.  Modelling cardiovascular physiological signals using adaptive Hermite and wavelet basis functions , 2010 .