Outliers and Random Noises in System Identification: a Compressed Sensing Approach

In this paper, we consider robust system identification under sparse outliers and random noises. In this problem, system parameters are observed through a Toeplitz matrix. All observations are subject to random noises and a few are corrupted with outliers. We reduce this problem of system identification to a sparse error correcting problem using a Toeplitz structured real-numbered coding matrix. We prove the performance guarantee of Toeplitz structured matrix in sparse error correction. Thresholds on the percentage of correctable errors for Toeplitz structured matrices are established. When both outliers and observation noise are present, we have shown that the estimation error goes to 0 asymptotically as long as the probability density function for observation noise is not "vanishing" around 0. No probabilistic assumptions are imposed on the outliers.

[1]  Biao Huang,et al.  System Identification , 2000, Control Theory for Physicists.

[2]  J. Kuelbs.  Probability on Banach spaces , 1978 .

[3]  Mihailo Stojnic,et al.  Various thresholds for ℓ1-optimization in compressed sensing , 2009, ArXiv.

[4]  Andrea Montanari,et al.  The Noise-Sensitivity Phase Transition in Compressed Sensing , 2010, IEEE Transactions on Information Theory.

[5]  D. Donoho,et al.  Neighborliness of randomly projected simplices in high dimensions. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[6]  David L. Donoho,et al.  High-Dimensional Centrally Symmetric Polytopes with Neighborliness Proportional to Dimension , 2006, Discret. Comput. Geom..

[7]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[8]  Cynthia Dwork,et al.  The price of privacy and the limits of LP decoding , 2007, STOC '07.

[9]  M. Rudelson,et al.  On sparse reconstruction from Fourier and Gaussian measurements , 2008 .

[10]  C. Xiru,et al.  Asymptotic Normality of Minimum L1-Norm Estimates in Linear Models , 1990 .

[11]  Emmanuel J. Candès,et al.  Highly Robust Error Correction byConvex Programming , 2006, IEEE Transactions on Information Theory.

[12]  Weiyu Xu,et al.  Compressed sensing - probabilistic analysis of a null-space characterization , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[13]  Holger Rauhut,et al.  Circulant and Toeplitz matrices in compressed sensing , 2009, ArXiv.

[14]  Weiyu Xu,et al.  Precise Stability Phase Transitions for $\ell_1$ Minimization: A Unified Geometric Framework , 2011, IEEE Transactions on Information Theory.

[15]  Yin Zhang,et al.  A Simple Proof for Recoverability of `1-Minimization , 2005 .

[16]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[17]  Weiyu Xu,et al.  Toeplitz matrix based sparse error correction in system identification: Outliers and random noises , 2012, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[18]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[19]  M. Ledoux.  The concentration of measure phenomenon , 2001 .

[20]  Stephen J. Wright,et al.  Toeplitz-Structured Compressed Sensing Matrices , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.

[21]  Colin McDiarmid,et al.  Surveys in Combinatorics, 1989: On the method of bounded differences , 1989 .

[22]  Justin K. Romberg,et al.  Compressive Sensing by Random Convolution , 2009, SIAM J. Imaging Sci..

[23]  Ao Tang,et al.  On state estimation with bad data detection , 2011, IEEE Conference on Decision and Control and European Control Conference.