Learning Nonlinear State-Space Models Using Deep Autoencoders

We introduce a new methodology for the identification of nonlinear state-space models using machine-learning techniques based on deep autoencoders for dimensionality reduction and neural networks. By learning a direct acyclic computational graph, our framework simultaneously identifies the nonlinear output and state-update maps, and optionally a neural state observer. After formulating the approach in detail and providing guidelines for tuning the related hyperparameters and reducing the model order, we show its capability of fitting a nonlinear model from an input/output dataset generated by a benchmark nonlinear system. Performance is assessed in terms of the ability of filtering and predicting output signals ahead, and of controlling the system via nonlinear model predictive control (MPC) based on the identified model.

[1]  David Q. Mayne,et al.  Model predictive control: Recent developments and future promise , 2014, Autom..

[2]  Yu Wang,et al.  A new concept using LSTM Neural Networks for dynamic system identification , 2017, 2017 American Control Conference (ACC).

[3]  Sanjiv Kumar,et al.  On the Convergence of Adam and Beyond , 2018 .

[4]  Alberto Bemporad,et al.  Piecewise affine regression via recursive multiple least squares and multicategory discrimination , 2016, Autom..

[5]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[6]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[7]  Alberto Bemporad,et al.  From linear to nonlinear MPC: bridging the gap via the real-time iteration , 2020, Int. J. Control.

[8]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[9]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[10]  Clarence W. Rowley,et al.  Variants of Dynamic Mode Decomposition: Boundary Condition, Koopman, and Fourier Analyses , 2012, J. Nonlinear Sci..

[11]  Lennart Ljung,et al.  Kernel methods in system identification, machine learning and function estimation: A survey , 2014, Autom..

[12]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[13]  Jean-Philippe Noël,et al.  Three Benchmarks Addressing Open Challenges in Nonlinear System Identification , 2017 .

[14]  Yoram Baram,et al.  Minimal order representation, estimation and feedback of continuous-time stochastic linear systems , 1984 .

[15]  Naoya Takeishi,et al.  Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition , 2017, NIPS.

[16]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.