On the stability of reproducing kernel Hilbert spaces of discrete-time impulse responses

Abstract Reproducing kernel Hilbert spaces (RKHSs) have proved themselves to be key tools for the development of powerful machine learning algorithms, the so-called regularized kernel-based approaches.Recently, they have also inspired the design of new linear system identification techniques able to challenge classical parametric prediction error methods. These facts motivate the study of the RKHS theory within the control community. In this note, we focus on the characterization of stable RKHSs, i.e. RKHSs of functions representing stable impulse responses. Related to this, working in an abstract functional analysis framework, Carmeli et al. (2006) has provided conditions for an RKHS to be contained in the classical Lebesgue spaces ℒ p . In particular, we specialize this analysis to the discrete-time case with p = 1 . The necessary and sufficient conditions for the stability of an RKHS are worked out by a quite simple proof, more easily accessible to the control community.

[1]  Francesco Dinuzzo,et al.  Kernels for Linear Time Invariant System Identification , 2012, SIAM J. Control. Optim..

[2]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[3]  Petre Stoica,et al.  Decentralized Control , 2018, The Control Systems Handbook.

[4]  Henrik Ohlsson,et al.  On the estimation of transfer functions, regularizations and Gaussian processes - Revisited , 2012, Autom..

[5]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[6]  G. Wahba,et al.  A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines , 1970 .

[7]  M. Bertero.  Linear Inverse and III-Posed Problems , 1989 .

[8]  Giuseppe De Nicolao,et al.  A new kernel-based approach for linear system identification , 2010, Autom..

[9]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[10]  S. Smale,et al.  Learning Theory Estimates via Integral Operators and Their Approximations , 2007 .

[11]  Milan Lukić,et al.  Stochastic processes with sample paths in reproducing kernel Hilbert spaces , 2001 .

[12]  A. Balakrishnan.  Applied Functional Analysis , 1976 .

[13]  Aleksandr Y. Aravkin,et al.  The Connection Between Bayesian Estimation of a Gaussian Random Field and RKHS , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Don R. Hush,et al.  An Explicit Description of the Reproducing Kernel Hilbert Spaces of Gaussian RBF Kernels , 2006, IEEE Transactions on Information Theory.

[15]  C. Carmeli,et al.  VECTOR VALUED REPRODUCING KERNEL HILBERT SPACES OF INTEGRABLE FUNCTIONS AND MERCER THEOREM , 2006 .

[16]  Robert E. Megginson.  An Introduction to Banach Space Theory , 1998 .

[17]  G. Wahba.  Spline models for observational data , 1990 .

[18]  Lennart Ljung,et al.  System Identification Via Sparse Multiple Kernel-Based Regularization Using Sequential Convex Optimization Techniques , 2014, IEEE Transactions on Automatic Control.

[19]  R. Kennedy,et al.  Hilbert Space Methods in Signal Processing , 2013 .

[20]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[21]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[22]  Alessandro Chiuso,et al.  Prediction error identification of linear systems: A nonparametric Gaussian regression approach , 2011, Autom..

[23]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[24]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .