Analysis of a nonsmooth optimization approach to robust estimation

In this paper, we consider the problem of identifying a linear map from measurements which are subject to intermittent and arbitrarily large errors. This is a fundamental problem in many estimation-related applications such as fault detection, state estimation in lossy networks, hybrid system identification, robust estimation, etc. The problem is hard because it exhibits some intrinsic combinatorial features. Therefore, obtaining an effective solution necessitates relaxations that are both solvable at a reasonable cost and effective in the sense that they can return the true parameter vector. The current paper discusses a nonsmooth convex optimization approach and provides a new analysis of its behavior. In particular, it is shown that under appropriate conditions on the data, an exact estimate can be recovered from data corrupted by a large (even infinite) number of gross errors.

[1]  René Vidal,et al.  Identification of Hybrid Systems: A Tutorial , 2007, Eur. J. Control.

[2]  Rama Chellappa,et al.  IEEE TRANSACTIONS ON SIGNAL PROCESSING 1 Analysis of Sparse Regularization Based Robust Regression Approaches , 2022 .

[3]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[4]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[5]  Laurent Bako,et al.  Recovering piecewise affine maps by sparse optimization , 2012 .

[6]  S. Sastry,et al.  An algebraic geometric approach to the identification of a class of linear hybrid systems , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[7]  Mario Sznaier,et al.  Hybrid system identification with faulty measurements and its application to activity analysis , 2011, IEEE Conference on Decision and Control and European Control Conference.

[8]  Duan Li,et al.  Reweighted 1-Minimization for Sparse Solutions to Underdetermined Linear Systems , 2012, SIAM J. Optim..

[9]  Henrik Ohlsson,et al.  Identification of switched linear regression models using sum-of-norms regularization , 2013, Autom..

[10]  Yoav Sharon,et al.  Minimum sum of distances estimator: Robustness and stability , 2009, 2009 American Control Conference.

[11]  René Vidal,et al.  A continuous optimization framework for hybrid system identification , 2011, Autom..

[12]  Peter J. Huber,et al.  The place of the L1-norm in robust estimation , 1987 .

[13]  R. Vidal A TUTORIAL ON SUBSPACE CLUSTERING , 2010 .

[14]  Weiyu Xu,et al.  System identification in the presence of outliers and random noises: A compressed sensing approach , 2014, Autom..

[15]  Weiyu Xu,et al.  Precise Stability Phase Transitions for $\ell_1$ Minimization: A Unified Geometric Framework , 2011, IEEE Transactions on Information Theory.

[16]  Paulo Tabuada,et al.  Secure Estimation and Control for Cyber-Physical Systems Under Adversarial Attacks , 2012, IEEE Transactions on Automatic Control.

[17]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[18]  J. Cadzow A Finite Algorithm for the Minimum $l_\infty $ Solution to a System of Consistent Linear Equations , 1973 .

[19]  Emmanuel J. Candès,et al.  Highly Robust Error Correction byConvex Programming , 2006, IEEE Transactions on Information Theory.

[20]  V. Yohai,et al.  Robust Statistics: Theory and Methods , 2006 .

[21]  Gérard Bloch,et al.  Selective $\ell_{1}$ Minimization for Sparse Recovery , 2014, IEEE Transactions on Automatic Control.

[22]  丸山 徹 Convex Analysisの二,三の進展について , 1977 .

[23]  R. Tibshirani The Lasso Problem and Uniqueness , 2012, 1206.0313.

[24]  Jacek Gondzio,et al.  Interior point methods 25 years later , 2012, Eur. J. Oper. Res..

[25]  Stéphane Lecoeuche,et al.  A sparse optimization approach to state observer design for switched linear systems , 2013, Syst. Control. Lett..

[26]  Xiaojun Chen,et al.  Convergence of Reweighted ' 1 Minimization Algorithms and Unique Solution of Truncated ' p Minimization , 2010 .

[27]  Biao Huang,et al.  System Identification , 2000, Control Theory for Physicists.

[28]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[29]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[30]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[31]  P. Rousseeuw Least Median of Squares Regression , 1984 .

[32]  A. Garulli,et al.  A survey on switched and piecewise affine system identification , 2012 .

[33]  Laurent Bako Subspace Clustering Through Parametric Representation and Sparse Optimization , 2014, IEEE Signal Processing Letters.

[34]  Constantino M. Lagoa,et al.  A Sparsification Approach to Set Membership Identification of Switched Affine Systems , 2012, IEEE Transactions on Automatic Control.

[35]  Laurent Bako,et al.  A Recursive Sparse Learning Method: Application to Jump Markov Linear Systems , 2011 .

[36]  Constantino M. Lagoa,et al.  A sparsification approach to set membership identification of a class of affine hybrid systems , 2008, 2008 47th IEEE Conference on Decision and Control.

[37]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[38]  Laurent Bako,et al.  Identification of switched linear systems via sparse optimization , 2011, Autom..

[39]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[40]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[41]  Alex Simpkins,et al.  System Identification: Theory for the User, 2nd Edition (Ljung, L.; 1999) [On the Shelf] , 2012, IEEE Robotics & Automation Magazine.

[42]  Michael B. Miller Linear Regression Analysis , 2013 .

[43]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[44]  F. Hampel A General Qualitative Definition of Robustness , 1971 .

[45]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[46]  Toshiharu Sugie,et al.  Identification of PWA models via data compression based on l1 optimization , 2011, IEEE Conference on Decision and Control and European Control Conference.

[47]  Stephen P. Boyd,et al.  Segmentation of ARX-models using sum-of-norms regularization , 2010, Autom..