Kernel-based Impulse Response Identification with Side-Information on Steady-State Gain

In this paper, we consider the problem of system identification when side-information is available on the steadystate (or DC) gain of the system. We formulate a general nonparametric identification method as an infinite-dimensional constrained convex program over the reproducing kernel Hilbert space (RKHS) of stable impulse responses. The objective function of this optimization problem is the empirical loss regularized with the norm of RKHS, and the constraint is considered for enforcing the integration of the steady-state gain side-information. The proposed formulation addresses both the discrete-time and continuous-time cases. We show that this program has a unique solution obtained by solving an equivalent finite-dimensional convex optimization. This solution has a closed-form when the empirical loss and regularization functions are quadratic and exact side-information is considered. We perform extensive numerical comparisons to verify the efficiency of the proposed identification methodology.

[1]  Roy S. Smith,et al.  Regularized Identification with Internal Positivity Side-Information , 2021, ArXiv.

[2]  Masaki Inoue,et al.  Subspace identification with moment matching , 2019, Autom..

[3]  Alberto De Santis,et al.  Identification of positive linear systems with Poisson output transformation , 2002, Autom..

[4]  Tianshi Chen,et al.  A shift in paradigm for system identification , 2019, Int. J. Control.

[5]  Ai Hui Tan,et al.  Kernel-Based Impulse Response Estimation With Prior DC Gain Using Built-In Self-Scaling Technique , 2020, IEEE Transactions on Instrumentation and Measurement.

[6]  Lucas P. R. K. Ihlenfeld,et al.  A novel subspace identification approach with passivity enforcement , 2021, Autom..

[7]  L. Ljung,et al.  On the Estimation of Transfer Functions, Regularizations and Gaussian Processes – Revisited , 2011 .

[8]  Raymond A. de Callafon,et al.  Subspace identification with eigenvalue constraints , 2013, Autom..

[9]  Bernhard Schölkopf,et al.  The representer theorem for Hilbert spaces: a necessary and sufficient condition , 2012, NIPS.

[10]  Pavel Trnka,et al.  Subspace like identification incorporating prior information , 2009, Autom..

[11]  Lennart Ljung,et al.  Kernel methods in system identification, machine learning and function estimation: A survey , 2014, Autom..

[12]  Parikshit Shah,et al.  Linear system identification via atomic norm regularization , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[13]  Johan Schoukens,et al.  Filter-based regularisation for impulse response modelling , 2016, ArXiv.

[14]  V. Yohai,et al.  Robust Statistics: Theory and Methods , 2006 .

[15]  Ahmed Mahmoud Abdelrahman Elanany.  Improved subspace identication with prior information using constrained least-squares , 2011 .

[16]  Lennart Ljung,et al.  Version 8 of the Matlab System Identification Toolbox , 2012 .

[17]  Roy S. Smith,et al.  Regularized System Identification: A Hierarchical Bayesian Approach , 2020 .

[18]  Lennart Ljung,et al.  Regularized linear system identification using atomic, nuclear and kernel-based norms: The role of the stability constraint , 2015, Autom..

[19]  Roy S. Smith,et al.  Nonlinear System Identification With Prior Knowledge on the Region of Attraction , 2020, IEEE Control Systems Letters.

[20]  Mohamed Darwish,et al.  The quest for the right kernel in Bayesian impulse response identification: The use of OBFs , 2018, Autom..

[21]  C. Carmeli,et al.  VECTOR VALUED REPRODUCING KERNEL HILBERT SPACES OF INTEGRABLE FUNCTIONS AND MERCER THEOREM , 2006 .

[22]  Mohammad Khosravi,et al.  Kernel-Based Identification of Positive Systems , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[23]  Tianshi Chen,et al.  On the stability of reproducing kernel Hilbert spaces of discrete-time impulse responses , 2018, Autom..

[24]  Giuseppe De Nicolao,et al.  A new kernel-based approach for linear system identification , 2010, Autom..

[25]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[26]  G. Marinoschi.  An identification problem , 2005 .

[27]  M. Morari,et al.  Closed-loop properties from steady-state gain information , 1985 .

[28]  Tor Arne Johansen,et al.  Constrained and regularized system identification , 1997 .

[29]  Yoshito Ohta,et al.  Bayesian positive system identification: Truncated Gaussian prior and hyperparameter estimation , 2021, Syst. Control. Lett..

[30]  Mohammad Khosravi,et al.  On Robustness of Kernel-Based Regularized System Identification , 2021, IFAC-PapersOnLine.

[31]  J. Peypouquet.  Convex Optimization in Normed Spaces: Theory, Methods and Examples , 2015 .

[32]  Amir Ali Ahmadi,et al.  Learning Dynamical Systems with Side Information (short version) , 2019 .

[33]  Giulio Bottegal,et al.  An empirical Bayes approach to identification of modules in dynamic networks , 2017, Autom..

[34]  H. Brezis.  Functional Analysis, Sobolev Spaces and Partial Differential Equations , 2010 .

[35]  Lennart Ljung,et al.  Handling Certain Structure Information in Subspace Identification , 2009 .

[36]  Toshiharu Sugie,et al.  Kernel-Based Impulse Response Estimation With a Priori Knowledge on the DC Gain , 2018, IEEE Control Systems Letters.

[37]  Masaki Inoue,et al.  Subspace identification method incorporated with a priori information characterized in frequency domain , 2016, 2016 European Control Conference (ECC).

[38]  Fredrik Lindsten,et al.  Bayesian nonparametric identification of Wiener systems , 2019, Autom..

[39]  Peter C. Young,et al.  Direct Identification of Continuous-time Models from Sampled Data: Issues, Basic Solutions and Relevance , 2008 .

[40]  S. O. Reza Moheimani,et al.  Estimation of phase constrained MIMO transfer functions with application to flexible structures with , 2005 .

[41]  Michel Verhaegen,et al.  Subspace identification of circulant systems , 2008, Autom..

[42]  Roy S. Smith,et al.  Low-Complexity Identification by Sparse Hyperparameter Estimation , 2020 .

[43]  Ivan Markovsky,et al.  Subspace identification with constraints on the impulse response , 2017, Int. J. Control.

[44]  Tianshi Chen,et al.  On kernel design for regularized LTI system identification , 2016, Autom..

[45]  Johan A. K. Suykens,et al.  Identification of stable models in subspace identification by using regularization , 2001, IEEE Trans. Autom. Control..

[46]  Roy S. Smith,et al.  Convex Nonparametric Formulation for Identification of Gradient Flows , 2020, IEEE Control Systems Letters.

[47]  Toshiharu Sugie,et al.  Extension of First-Order Stable Spline Kernel to Encode Relative Degree , 2017 .

[48]  Xu Cheng,et al.  Robust stability constrained model predictive control , 2004, Proceedings of the 2004 American Control Conference.

[49]  Roy S. Smith,et al.  Frequency Domain Subspace Identification Using Nuclear Norm Minimization and Hankel Matrix Realizations , 2014, IEEE Transactions on Automatic Control.

[50]  Johan A. K. Suykens,et al.  Identification of positive real models in subspace identification by using regularization , 2003, IEEE Trans. Autom. Control..

[51]  D.S. Bernstein,et al.  Subspace identification with lower bounded modal frequencies , 2004, Proceedings of the 2004 American Control Conference.

[52]  Roy S. Smith,et al.  Kernel-Based Identification with Frequency Domain Side-Information , 2021, ArXiv.

[53]  Dennis S. Bernstein,et al.  Subspace identification with guaranteed stability using constrained optimization , 2003, IEEE Trans. Autom. Control..

[54]  Marion Gilson,et al.  CONTSID: a Matlab toolbox for standard and advanced identification of black-box continuous-time models , 2018 .

[55]  Paul Tseng,et al.  Hankel Matrix Rank Minimization with Applications to System Identification and Realization , 2013, SIAM J. Matrix Anal. Appl..

[56]  KrauseAndreas,et al.  Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting , 2012 .

[57]  Giulio Bottegal,et al.  A nonparametric kernel-based approach to Hammerstein system identification , 2017, Autom..

[58]  Lorenzo Farina,et al.  On model consistency in compartmental systems identification , 2002, Autom..

[59]  P. J. Campo,et al.  Achievable closed-loop properties of systems under decentralized control: conditions involving the steady-state gain , 1994, IEEE Trans. Autom. Control..

[60]  Masaki Inoue,et al.  System identification method inheriting steady-state characteristics of existing model , 2019, Int. J. Control.

[61]  Ian R. Manchester,et al.  Specialized Interior-Point Algorithm for Stable Nonlinear System Identification , 2018, IEEE Transactions on Automatic Control.