On Robustness of Kernel-Based Regularized System Identification

This paper presents a novel feature of the kernel-based system identification method. We prove that the regularized kernel-based approach for the estimation of a finite impulse response is equivalent to a robust least-squares problem with a particular uncertainty set defined in terms of the kernel matrix, and thus, it is called kernel-based uncertainty set. We provide a theoretical foundation for the robustness of the kernel-based approach to input disturbances. Based on robust and regularized least-squares methods, different formulations of system identification are considered, where the kernel-based uncertainty set is employed in some of them. We apply these methods to a case where the input measurements are subject to disturbances. Subsequently, we perform extensive numerical experiments and compare the results to examine the impact of utilizing kernel-based uncertainty sets in the identification procedure. The numerical experiments confirm that the robust least square identification approach with the kernel-based uncertainty set improves the robustness of the estimation to the input disturbances.

[1]  Roy S. Smith,et al.  Convex Nonparametric Formulation for Identification of Gradient Flows , 2020, IEEE Control Systems Letters.

[2]  Lennart Ljung,et al.  Regularized LTI System Identification with Multiple Regularization Matrix , 2018 .

[3]  Toshiharu Sugie,et al.  Kernel-based system identification improving robustness to input disturbances: A preliminary study on its application to AR models , 2020, 2020 European Control Conference (ECC).

[4]  Lennart Ljung,et al.  System Identification Via Sparse Multiple Kernel-Based Regularization Using Sequential Convex Optimization Techniques , 2014, IEEE Transactions on Automatic Control.

[5]  Tianshi Chen,et al.  A shift in paradigm for system identification , 2019, Int. J. Control.

[6]  Roy S. Smith,et al.  Regularized System Identification: A Hierarchical Bayesian Approach , 2020 .

[7]  Alessandro Chiuso,et al.  The harmonic analysis of kernel functions , 2017, Autom..

[8]  Roy S. Smith,et al.  Frequency Domain Subspace Identification Using Nuclear Norm Minimization and Hankel Matrix Realizations , 2014, IEEE Transactions on Automatic Control.

[9]  Yoshito Ohta,et al.  Positive FIR System Identification using Maximum Entropy Prior , 2018 .

[10]  Giuseppe De Nicolao,et al.  A new kernel-based approach for linear system identification , 2010, Autom..

[11]  G. Marinoschi An identification problem , 2005 .

[12]  Roy S. Smith,et al.  Low-Complexity Identification by Sparse Hyperparameter Estimation , 2020 .

[13]  Fredrik Lindsten,et al.  Bayesian nonparametric identification of Wiener systems , 2019, Autom..

[14]  Toshiharu Sugie,et al.  Kernel-Based Impulse Response Estimation With a Priori Knowledge on the DC Gain , 2018, IEEE Control Systems Letters.

[15]  Giulio Bottegal,et al.  A nonparametric kernel-based approach to Hammerstein system identification , 2017, Autom..

[16]  Alessandro Chiuso,et al.  Regularization and Bayesian learning in dynamical systems: Past, present and future , 2015, Annu. Rev. Control..

[17]  Parikshit Shah,et al.  Linear system identification via atomic norm regularization , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[18]  Johan Schoukens,et al.  Filter-based regularisation for impulse response modelling , 2016, ArXiv.

[19]  Roy S. Smith,et al.  Nonlinear System Identification With Prior Knowledge on the Region of Attraction , 2020, IEEE Control Systems Letters.

[20]  Mohammad Khosravi,et al.  Kernel-Based Identification of Positive Systems , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[21]  N. Aronszajn Theory of Reproducing Kernels. , 1950 .

[22]  Laurent El Ghaoui,et al.  Robust Solutions to Least-Squares Problems with Uncertain Data , 1997, SIAM J. Matrix Anal. Appl..

[23]  L. Ljung,et al.  Asymptotic Properties of Generalized Cross Validation Estimators for Regularized System Identification , 2018 .

[24]  Linear Time-Periodic System Identification with Grouped Atomic Norm Regularization , 2020, IFAC-PapersOnLine.

[25]  Tianshi Chen,et al.  Multiple Kernel Based Regularized System Identification with SURE Hyper-parameter Estimator , 2018 .

[26]  Lennart Ljung,et al.  Kernel methods in system identification, machine learning and function estimation: A survey , 2014, Autom..

[27]  Giulio Bottegal,et al.  Local Module Identification in Dynamic Networks Using Regularized Kernel-Based Methods , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[28]  Tianshi Chen,et al.  On kernel design for regularized LTI system identification , 2016, Autom..

[29]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .