On Input Design for Regularized LTI System Identification: Power-constrained Input

Input design is an important issue for classical system identification methods but has not been investigated for the kernel-based regularization method (KRM) until very recently. In this paper, we consider in the time domain the input design problem of KRMs for LTI system identification. Different from the recent result, we adopt a Bayesian perspective and in particular make use of scalar measures (e.g., the $A$-optimality, $D$-optimality, and $E$-optimality) of the Bayesian mean square error matrix as the design criteria subject to power-constraint on the input. Instead to solve the optimization problem directly, we propose a two-step procedure. In the first step, by making suitable assumptions on the unknown input, we construct a quadratic map (transformation) of the input such that the transformed input design problems are convex, the number of optimization variables is independent of the number of input data, and their global minima can be found efficiently by applying well-developed convex optimization software packages. In the second step, we derive the expression of the optimal input based on the global minima found in the first step by solving the inverse image of the quadratic map. In addition, we derive analytic results for some special types of fixed kernels, which provide insights on the input design and also its dependence on the kernel structure.

[1]  A. Brøndsted An Introduction to Convex Polytopes , 1982 .

[2]  Francesca P. Carli,et al.  Maximum Entropy Kernels for System Identification , 2014, IEEE Transactions on Automatic Control.

[3]  Alessandro Chiuso,et al.  Prediction error identification of linear systems: A nonparametric Gaussian regression approach , 2011, Autom..

[4]  Håkan Hjalmarsson,et al.  Least-squares estimation of a class of frequency functions: A finite sample variance expression , 2006, Autom..

[5]  Mohamed Darwishy,et al.  Perspectives of orthonormal basis functions based kernels in Bayesian system identification? , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[6]  Lennart Ljung,et al.  On Asymptotic Properties of Hyperparameter Estimators for Kernel-based Regularization Methods , 2017, Autom..

[7]  H. Jansson Experiment design with applications in identification for control , 2004 .

[8]  Henrik Ohlsson,et al.  On the estimation of transfer functions, regularizations and Gaussian processes - Revisited , 2012, Autom..

[9]  Tianshi Chen,et al.  On kernel design for regularized LTI system identification , 2016, Autom..

[10]  Håkan Hjalmarsson,et al.  System identification of complex and structured systems , 2009, 2009 European Control Conference (ECC).

[11]  Alessandro Chiuso,et al.  Regularization and Bayesian learning in dynamical systems: Past, present and future , 2015, Annu. Rev. Control..

[12]  Robert M. Gray,et al.  Toeplitz and Circulant Matrices: A Review , 2005, Found. Trends Commun. Inf. Theory.

[13]  Lennart Ljung,et al.  Regularized linear system identification using atomic, nuclear and kernel-based norms: The role of the stability constraint , 2015, Autom..

[14]  Duan Li,et al.  Simultaneous Diagonalization of Matrices and Its Applications in Quadratically Constrained Quadratic Programming , 2015, SIAM J. Optim..

[15]  Robert Luxmore Payne,et al.  Optimal experiment design for dynamic system identification , 1974 .

[16]  Michel Gevers,et al.  Identification For Control: Optimal Input Design With Respect To A Worst-Case $\nu$-gap Cost Function , 2002, SIAM J. Control. Optim..

[17]  D.G. Dudley,et al.  Dynamic system identification experiment design and data analysis , 1979, Proceedings of the IEEE.

[18]  Håkan Hjalmarsson,et al.  From experiment design to closed-loop control , 2005, Autom..

[19]  Lennart Ljung,et al.  Regularized system identification using orthonormal basis functions , 2015, 2015 European Control Conference (ECC).

[20]  Toshiharu Sugie,et al.  Informative input design for Kernel-Based system identification , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[21]  Lennart Ljung,et al.  System Identification Via Sparse Multiple Kernel-Based Regularization Using Sequential Convex Optimization Techniques , 2014, IEEE Transactions on Automatic Control.

[22]  Lennart Ljung,et al.  System identification toolbox for use with MATLAB , 1988 .

[23]  Håkan Hjalmarsson,et al.  Input design via LMIs admitting frequency-wise model specifications in confidence regions , 2005, IEEE Transactions on Automatic Control.

[24]  L. Ljung,et al.  Regularization Features in the System Identification Toolbox , 2015 .

[25]  Lennart Ljung,et al.  Maximum entropy properties of discrete-time first-order stable spline kernel , 2014, Autom..

[26]  Xavier Bombois,et al.  Least costly identification experiment for control , 2006, Autom..

[27]  Michel Gevers,et al.  Identification for Control: From the Early Achievements to the Revival of Experiment Design , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[28]  Gérard Meurant,et al.  A Review on the Inverse of Symmetric Tridiagonal and Block Tridiagonal Matrices , 1992, SIAM J. Matrix Anal. Appl..

[29]  Lennart Ljung,et al.  Kernel methods in system identification, machine learning and function estimation: A survey , 2014, Autom..

[30]  Biao Huang,et al.  System Identification , 2000, Control Theory for Physicists.

[31]  G. Tee Eigenvectors of block circulant and alternating circulant matrices , 2005 .

[32]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[33]  Giuseppe De Nicolao,et al.  A new kernel-based approach for linear system identification , 2010, Autom..

[34]  Alessandro Chiuso,et al.  The harmonic analysis of kernel functions , 2017, Autom..

[35]  Graham C. Goodwin,et al.  Estimated Transfer Functions with Application to Model Order Selection , 1992 .

[36]  Raman K. Mehra,et al.  Optimal input signals for parameter estimation in dynamic systems--Survey and new results , 1974 .

[37]  Johan Schoukens,et al.  Filter-based regularisation for impulse response modelling , 2016, ArXiv.

[38]  Alessandro Chiuso,et al.  Tuning complexity in regularized kernel-based regression and linear system identification: The robustness of the marginal likelihood estimator , 2015, Autom..

[39]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.