Recursive nonlinear-system identification using latent variables

In this paper we develop a method for learning nonlinear systems with multiple outputs and inputs. We begin by modelling the errors of a nominal predictor of the system using a latent variable framework. Then using the maximum likelihood principle we derive a criterion for learning the model. The resulting optimization problem is tackled using a majorization-minimization approach. Finally, we develop a convex majorization technique and show that it enables a recursive identification method. The method learns parsimonious predictive models and is tested on both synthetic and real nonlinear systems.

[1]  Per Mattsson,et al.  Convergence analysis for recursive Hammerstein identification , 2016, Autom..

[2]  S. Billings.  Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Temporal Domains , 2013 .

[3]  A. Belloni,et al.  Square-Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming , 2011 .

[4]  E. Bai,et al.  Block Oriented Nonlinear System Identification , 2010 .

[5]  K. Lange,et al.  The MM Alternative to EM , 2010, 1104.2203.

[6]  Lennart Ljung,et al.  Theory and Practice of Recursive Identification , 1983 .

[7]  Alberto Bemporad,et al.  A bounded-error approach to piecewise affine system identification , 2005, IEEE Transactions on Automatic Control.

[8]  Petre Stoica,et al.  Exact initialization of the recursive least-squares algorithm , 2002 .

[9]  Gianluigi Pillonetto,et al.  Consistent identification of Wiener systems: A machine learning viewpoint , 2013, Autom..

[10]  Er-Wei Bai,et al.  Recursive Direct Weight Optimization in Nonlinear System Identification: A Minimal Probability Approach , 2007, IEEE Transactions on Automatic Control.

[11]  C. Abdallah,et al.  Recent techniques for the identification of piecewise affine and hybrid systems , 2006 .

[12]  Lennart Ljung,et al.  Nonlinear black-box modeling in system identification: a unified overview , 1995, Autom..

[13]  Lennart Ljung,et al.  System identification toolbox for use with MATLAB , 1988 .

[14]  Henrik Ohlsson,et al.  Identification of switched linear regression models using sum-of-norms regularization , 2013, Autom..

[15]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[16]  Arno Solin,et al.  Hilbert space methods for reduced-rank Gaussian process regression , 2014, Stat. Comput..

[17]  Petre Stoica,et al.  Online Hyperparameter-Free Sparse Estimation Method , 2015, IEEE Transactions on Signal Processing.

[18]  Johan Schoukens,et al.  Convergence analysis and experiments using an RPEM based on nonlinear ODEs and midpoint integration , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[19]  P. V. D. Hof,et al.  System identification with generalized orthonormal basis functions , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[20]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[21]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[22]  D. Hunter,et al.  A Tutorial on MM Algorithms , 2004 .

[23]  Jacob Roll,et al.  Nonlinear system identification via direct weight optimization , 2005, Autom..

[24]  Jan Swevers,et al.  Identification of nonlinear systems using Polynomial Nonlinear State Space models , 2010, Autom..

[25]  Jian Li,et al.  Weighted SPICE: A unifying approach for hyperparameter-free sparse estimation , 2014, Digit. Signal Process..

[26]  David Barber,et al.  Bayesian reasoning and machine learning , 2012 .

[27]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[28]  Lennart Ljung,et al.  Kernel methods in system identification, machine learning and function estimation: A survey , 2014, Autom..

[29]  M. Enqvist.  Linear models of nonlinear systems , 2005 .

[30]  A. Juloski,et al.  Data-based hybrid modelling of the component placement process in pick-and-place machines , 2004 .

[31]  Huixin Chen.  Extended recursive least squares algorithm for nonlinear stochastic systems , 2004, Proceedings of the 2004 American Control Conference.

[32]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[33]  Lennart Ljung,et al.  Model Validation and Model Error Modeling , 1999 .

[34]  Torbjörn Wigren,et al.  Recursive prediction error identification using the nonlinear wiener model , 1993, Autom..

[35]  Rik Pintelon,et al.  Linear System Identification in a Nonlinear Setting: Nonparametric Analysis of the Nonlinear Distortions and Their Impact on the Best Linear Approximation , 2016, IEEE Control Systems.

[36]  Petre Stoica,et al.  Decentralized Control , 2018, The Control Systems Handbook.

[37]  Thomas B. Schön,et al.  Online sparse Gaussian process regression using FITC and PITC approximations , 2015 .

[38]  Torbjörn Wigren,et al.  Recursive prediction error identification and scaling of non-linear state space models using a restricted black box parameterization , 2006, Autom..