Kernel-Based Identification with Frequency Domain Side-Information

This paper discusses the problem of system identification when frequency domain side-information is available. Initially, we consider the case where the side-information is provided as the H∞-norm of the system being bounded by a given scalar. This framework allows considering different forms of frequency domain side-information, such as the dissipativity of the system. We propose a nonparametric identification approach for estimating the impulse response of the system under the given side-information. The estimation problem is formulated as a constrained optimization in a stable reproducing kernel Hilbert space, where suitable constraints are considered for incorporating the desired frequency domain features. The resulting optimization has an infinite-dimensional feasible set with an infinite number of constraints. We show that this problem is a well-defined convex program with a unique solution. We propose a heuristic that tightly approximates this unique solution. The proposed approach is equivalent to solving a finite-dimensional convex quadratically constrained quadratic program. The efficiency of the discussed method is verified by several numerical examples.

[1]  Roy S. Smith,et al.  Regularized System Identification: A Hierarchical Bayesian Approach , 2020 .

[2]  Lennart Ljung,et al.  Perspectives on system identification , 2010, Annu. Rev. Control..

[3]  Peter C. Young,et al.  Direct Identification of Continuous-time Models from Sampled Data: Issues, Basic Solutions and Relevance , 2008 .

[4]  Ahmed Mahmoud Abdelrahman Elanany.  Improved subspace identication with prior information using constrained least-squares , 2011 .

[5]  M. Fazel,et al.  Reweighted nuclear norm minimization with application to system identification , 2010, Proceedings of the 2010 American Control Conference.

[6]  Lennart Ljung,et al.  Version 8 of the Matlab System Identification Toolbox , 2012 .

[7]  Athanasios C. Antoulas,et al.  Approximation of Large-Scale Dynamical Systems , 2005, Advances in Design and Control.

[8]  Lennart Ljung,et al.  Regularized linear system identification using atomic, nuclear and kernel-based norms: The role of the stability constraint , 2015, Autom..

[9]  Mohammad Khosravi,et al.  On Robustness of Kernel-Based Regularized System Identification , 2021, IFAC-PapersOnLine.

[10]  Roy S. Smith,et al.  Convex Nonparametric Formulation for Identification of Gradient Flows , 2020, IEEE Control Systems Letters.

[11]  Parikshit Shah,et al.  Linear system identification via atomic norm regularization , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[12]  Johan Schoukens,et al.  Filter-based regularisation for impulse response modelling , 2016, ArXiv.

[13]  Roy S. Smith,et al.  Nonlinear System Identification With Prior Knowledge on the Region of Attraction , 2020, IEEE Control Systems Letters.

[14]  Yoshito Ohta,et al.  Positive FIR System Identification using Maximum Entropy Prior , 2018 .

[15]  Alessandro Chiuso,et al.  System Identification: A Machine Learning Perspective , 2019, Annu. Rev. Control. Robotics Auton. Syst..

[16]  Mohammad Khosravi,et al.  Kernel-Based Identification of Positive Systems , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[17]  Roy S. Smith,et al.  Regularized Identification with Internal Positivity Side-Information , 2021, ArXiv.

[18]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[19]  Ian R. Manchester,et al.  Scalable identification of stable positive systems , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[20]  Amir Ali Ahmadi,et al.  Learning Dynamical Systems with Side Information (short version) , 2019 .

[21]  Simone Formentin,et al.  Kernel-based identification of asymptotically stable continuous-time linear dynamical systems , 2021, Int. J. Control.

[22]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[23]  Bernhard Schölkopf,et al.  The representer theorem for Hilbert spaces: a necessary and sufficient condition , 2012, NIPS.

[24]  Lorenzo Farina,et al.  On model consistency in compartmental systems identification , 2002, Autom..

[25]  J. Willems.  Dissipative dynamical systems part I: General theory , 1972 .

[26]  C. Carmeli,et al.  VECTOR VALUED REPRODUCING KERNEL HILBERT SPACES OF INTEGRABLE FUNCTIONS AND MERCER THEOREM , 2006 .

[27]  Masaki Inoue,et al.  System identification method inheriting steady-state characteristics of existing model , 2019, Int. J. Control.

[28]  G. Zames.  On the input-output stability of time-varying nonlinear feedback systems Part one: Conditions derived using concepts of loop gain, conicity, and positivity , 1966 .

[29]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[30]  Masaki Inoue,et al.  Subspace identification with moment matching , 2019, Autom..

[31]  Frank Allgöwer,et al.  Provably Robust Verification of Dissipativity Properties from Data , 2020, IEEE Transactions on Automatic Control.

[32]  Alberto De Santis,et al.  Identification of positive linear systems with Poisson output transformation , 2002, Autom..

[33]  Lennart Ljung,et al.  Kernel methods in system identification, machine learning and function estimation: A survey , 2014, Autom..

[34]  Tianshi Chen,et al.  A shift in paradigm for system identification , 2019, Int. J. Control.

[35]  Ian R. Manchester,et al.  Identification of externally positive systems , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[36]  Tianshi Chen,et al.  On kernel design for regularized LTI system identification , 2016, Autom..

[37]  J. Peypouquet.  Convex Optimization in Normed Spaces: Theory, Methods and Examples , 2015 .

[38]  Ian R. Manchester,et al.  Specialized Interior-Point Algorithm for Stable Nonlinear System Identification , 2018, IEEE Transactions on Automatic Control.

[39]  T. Sugie,et al.  Subspace system identification considering both noise attenuation and use of prior knowledge , 1996, Proceedings of 35th IEEE Conference on Decision and Control.

[40]  Keita Hara,et al.  Learning Koopman Operator under Dissipativity Constraints , 2019, IFAC-PapersOnLine.

[41]  Dennis S. Bernstein,et al.  Subspace identification with guaranteed stability using constrained optimization , 2003, IEEE Trans. Autom. Control..

[42]  D. Bernstein,et al.  First-order-hold sampling of positive real systems and subspace identification of positive real models , 2004, Proceedings of the 2004 American Control Conference.

[43]  Marion Gilson,et al.  CONTSID: a Matlab toolbox for standard and advanced identification of black-box continuous-time models , 2018 .

[44]  Paul Tseng,et al.  Hankel Matrix Rank Minimization with Applications to System Identification and Realization , 2013, SIAM J. Matrix Anal. Appl..

[45]  KrauseAndreas,et al.  Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting , 2012 .

[46]  Roy S. Smith,et al.  Frequency Domain Subspace Identification Using Nuclear Norm Minimization and Hankel Matrix Realizations , 2014, IEEE Transactions on Automatic Control.

[47]  Frank Allgöwer,et al.  One-Shot Verification of Dissipativity Properties From Input–Output Data , 2019, IEEE Control Systems Letters.

[48]  Johan A. K. Suykens,et al.  Identification of positive real models in subspace identification by using regularization , 2003, IEEE Trans. Autom. Control..

[49]  Jan Maximilian Montenbruck,et al.  Sampling Strategies for Data-Driven Inference of Input–Output System Properties , 2019, IEEE Transactions on Automatic Control.

[50]  Roy S. Smith,et al.  Low-Complexity Identification by Sparse Hyperparameter Estimation , 2020 .

[51]  Tianshi Chen,et al.  On the stability of reproducing kernel Hilbert spaces of discrete-time impulse responses , 2018, Autom..

[52]  Roy S. Smith,et al.  Kernel-based Impulse Response Identification with Side-Information on Steady-State Gain , 2021, ArXiv.

[53]  Frank Allgöwer,et al.  Determining dissipation inequalities from input-output samples , 2017 .

[54]  Masaki Inoue,et al.  Subspace identification method incorporated with a priori information characterized in frequency domain , 2016, 2016 European Control Conference (ECC).

[55]  Alexandre Proutière,et al.  A stochastic multi-armed bandit approach to nonparametric H∞-norm estimation , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[56]  Raymond A. de Callafon,et al.  Subspace identification with eigenvalue constraints , 2013, Autom..

[57]  Giuseppe De Nicolao,et al.  A new kernel-based approach for linear system identification , 2010, Autom..

[58]  G. Marinoschi.  An identification problem , 2005 .