Sparse Identification of Nonlinear Functions and Parametric Set Membership Optimality Analysis

Sparse identification can be relevant in the automatic control field to solve several problems for nonlinear systems such as identification, control, filtering, fault detection. However, identifying a maximally sparse approximation of a nonlinear function is in general an NP-hard problem. The common approach is to use relaxed or greedy algorithms that, under certain conditions, can provide sparsest solutions. In this technical note, a combined l1-relaxed-greedy algorithm is proposed and conditions are given, under which the approximation derived by the algorithm is a sparsest one. Differently from other conditions available in the literature, the ones provided here can be actually verified for any choice of the basis functions defining the sparse approximation. A Set Membership analysis is also carried out, assuming that the function to approximate is a linear combination of unknown basis functions belonging to a known set of functions. It is shown that the algorithm is able to exactly select the basis functions which define the unknown function and to provide an optimal estimate of their coefficients.

[1]  B. Pasik-Duncan.  Control-oriented system identification: An H∞ approach , 2002 .

[2]  Nacim Meslem,et al.  Set-Membership Identification of Continuous-Time Systems , 2006 .

[3]  Constantino M. Lagoa,et al.  A Sparsification Approach to Set Membership Identification of Switched Affine Systems , 2012, IEEE Transactions on Automatic Control.

[4]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[5]  Carlo Novara,et al.  Model quality in identification of nonlinear systems , 2005, IEEE Transactions on Automatic Control.

[6]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[7]  Alexander B. Kurzhanski,et al.  Modeling Techniques for Uncertain Systems , 1994 .

[8]  Mario Sznaier,et al.  Sequential sparsification for change detection , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Jonathan R. Partington,et al.  Interpolation, identification, and sampling , 1997 .

[10]  Antonio Vicino,et al.  Optimal estimation theory for dynamic systems with set membership uncertainty: An overview , 1991, Autom..

[11]  Stephen P. Boyd,et al.  Graph Implementations for Nonsmooth Convex Programs , 2008, Recent Advances in Learning and Control.

[12]  Fred C. Schweppe,et al.  Uncertain dynamic systems , 1973 .

[13]  Carlo Novara,et al.  Set Membership identification of nonlinear systems , 2004, Autom..

[14]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[15]  Barak A. Pearlmutter,et al.  Blind Source Separation by Sparse Decomposition in a Signal Dictionary , 2001, Neural Computation.

[16]  Alfred O. Hero,et al.  Sparse LMS for system identification , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[17]  Lennart Ljung,et al.  Nonlinear black-box modeling in system identification: a unified overview , 1995, Autom..

[18]  Pierre Vandergheynst,et al.  A simple test to check the optimality of a sparse signal approximation , 2006, Signal Process..

[19]  Chao Feng,et al.  Hybrid system identification via sparse polynomial optimization , 2010, Proceedings of the 2010 American Control Conference.

[20]  Mario Milanese,et al.  Properties of least squares estimates in set membership identification, , 1995, Autom..

[21]  Jürgen Ackermann,et al.  Robust control prevents car skidding , 1997 .

[22]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[23]  Masayoshi Tomizuka,et al.  Design of simultaneously stabilizing controllers and its application to fault-tolerant lane-keeping controller design for automated vehicles , 2004, IEEE Transactions on Control Systems Technology.

[24]  Anton van Zanten,et al.  VDC, The Vehicle Dynamics Control System of Bosch , 1995 .

[25]  Carlo Novara.  Sparse identification of nonlinear functions and parametric Set Membership optimality analysis , 2011 .

[26]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[27]  Rémi Gribonval,et al.  Harmonic decomposition of audio signals with matching pursuit , 2003, IEEE Trans. Signal Process..

[28]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[29]  Biao Huang,et al.  System Identification , 2000, Control Theory for Physicists.

[30]  A. Atkinson.  Subset Selection in Regression , 1992 .

[31]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[32]  Mario Milanese,et al.  Yaw Stability Control Design Through a Mixed-Sensitivity Approach , 2009, IEEE Transactions on Control Systems Technology.

[33]  Justin K. Romberg,et al.  Beyond Nyquist: Efficient Sampling of Sparse Bandlimited Signals , 2009, IEEE Transactions on Information Theory.

[34]  Shili Lin,et al.  Sparse Support Vector Machines with L_{p} Penalty for Biomarker Identification , 2010, TCBB.

[35]  Rajesh Rajamani,et al.  Vehicle dynamics and control , 2005 .

[36]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[37]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[38]  Jean-Jacques Fuchs,et al.  Recovery of exact sparse representations in the presence of bounded noise , 2005, IEEE Transactions on Information Theory.

[39]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[40]  Federico Girosi,et al.  An Equivalence Between Sparse Approximation and Support Vector Machines , 1998, Neural Computation.

[41]  J. Norton,et al.  Bounding Approaches to System Identification , 1996 .

[42]  Mario Sznaier,et al.  Risk Adjusted Set Membership Identification of Wiener Systems , 2009, IEEE Transactions on Automatic Control.

[43]  Carlo Novara,et al.  Parametric identification of structured nonlinear systems , 2011, Autom..

[44]  Pascal Frossard,et al.  A posteriori quantization of progressive matching pursuit streams , 2004, IEEE Transactions on Signal Processing.

[45]  Tyrone L. Vincent,et al.  Concentration of measure inequalities for compressive toeplitz matrices with applications to detection and system identification , 2010, 49th IEEE Conference on Decision and Control (CDC).

[46]  Constantino M. Lagoa,et al.  A sparsification approach to set membership identification of a class of affine hybrid systems , 2008, 2008 47th IEEE Conference on Decision and Control.