Sparse linear regression from perturbed data

The problem of sparse linear regression is relevant in the context of linear system identification from large datasets. When data are collected from real-world experiments, measurements are always affected by perturbations or low-precision representations. However, the problem of sparse linear regression from fully-perturbed data is scarcely studied in the literature, due to its mathematical complexity. In this paper, we show that, by assuming bounded perturbations, this problem can be tackled by solving low-complex l2 and l1 minimization problems. Both theoretical guarantees and numerical results are illustrated in the paper.

[1]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[2]  Avishy Carmi,et al.  Compressive system identification: Sequential methods and entropy bounds , 2013, Digit. Signal Process..

[3]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[4]  Steven L. Brunton,et al.  Data-Driven Science and Engineering , 2019 .

[5]  Sophie Fosson,et al.  Sparsity-promoting sensor selection with energy harvesting constraints , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  J. Lasserre.  An Introduction to Polynomial and Semi-Algebraic Optimization , 2015 .

[7]  Enrico Magli,et al.  Distributed ADMM for in-network reconstruction of sparse signals with innovations , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[8]  Jean-Jacques Fuchs,et al.  On sparse representations in arbitrary redundant bases , 2004, IEEE Transactions on Information Theory.

[9]  Cishen Zhang,et al.  Robustly Stable Signal Recovery in Compressed Sensing With Structured Matrix Perturbation , 2011, IEEE Transactions on Signal Processing.

[10]  Enrico Magli,et al.  Steerable Discrete Cosine Transform , 2017, IEEE Trans. Image Process..

[11]  Shahrokh Valaee,et al.  Multiple Target Localization Using Compressive Sensing , 2009, GLOBECOM 2009 - 2009 IEEE Global Telecommunications Conference.

[12]  Phillipp Kaestner,et al.  Linear And Nonlinear Programming , 2016 .

[13]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[14]  Torsten Söderström,et al.  Errors-in-variables methods in system identification , 2018, Autom..

[15]  Georgios B. Giannakis,et al.  Sparsity-Cognizant Total Least-Squares for Perturbed Compressive Sampling , 2010, IEEE Transactions on Signal Processing.

[16]  Enrico Magli,et al.  Block-sparsity-based localization in wireless sensor networks , 2015, EURASIP J. Wirel. Commun. Netw..

[17]  Tyrone L. Vincent,et al.  Compressive System Identification in the Linear Time-Invariant framework , 2011, IEEE Conference on Decision and Control and European Control Conference.

[18]  A. Richards,et al.  Robust model predictive control with imperfect information , 2005, Proceedings of the 2005, American Control Conference, 2005..

[19]  Sabine Van Huffel,et al.  Overview of total least-squares methods , 2007, Signal Process..

[20]  Jean-Jacques Fuchs,et al.  Recovery of exact sparse representations in the presence of bounded noise , 2005, IEEE Transactions on Information Theory.

[21]  Michael Elad,et al.  Optimized Projections for Compressed Sensing , 2007, IEEE Transactions on Signal Processing.

[22]  Alberto Bemporad,et al.  Robust model predictive control: A survey , 1998, Robustness in Identification and Control.

[23]  Jean B. Lasserre,et al.  Global Optimization with Polynomials and the Problem of Moments , 2000, SIAM J. Optim..

[24]  Vito Cerone,et al.  A linear programming approach to sparse linear regression with quantized data , 2019, 2019 American Control Conference (ACC).

[25]  Håkan Hjalmarsson,et al.  Sparse Estimation of Polynomial and Rational Dynamical Models , 2014, IEEE Transactions on Automatic Control.

[26]  Dario Piga,et al.  Set-Membership Error-in-Variables Identification Through Convex Relaxation Techniques , 2012, IEEE Transactions on Automatic Control.

[27]  Gene H. Golub,et al.  An analysis of the total least squares problem , 1980, Milestones in Matrix Computation.

[28]  Aggelos K. Katsaggelos,et al.  Construction of Incoherent Unit Norm Tight Frames With Application to Compressed Sensing , 2014, IEEE Transactions on Information Theory.

[29]  Yuantao Gu,et al.  $l_{0}$ Norm Constraint LMS Algorithm for Sparse System Identification , 2009, IEEE Signal Processing Letters.

[30]  Vito Cerone,et al.  Feasible parameter set for linear models with bounded errors in all variables , 1993, Autom..

[31]  Salar Fattahi,et al.  Data-Driven Sparse System Identification , 2018, 2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[32]  Thomas Strohmer,et al.  General Deviants: An Analysis of Perturbations in Compressed Sensing , 2009, IEEE Journal of Selected Topics in Signal Processing.

[33]  Giorgio Battistelli,et al.  Moving-horizon state estimation for nonlinear discrete-time systems: New stability results and approximation schemes , 2008, Autom..

[34]  Gene H. Golub,et al.  Some modified matrix eigenvalue problems , 1973, Milestones in Matrix Computation.

[35]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[36]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[37]  Somantika Datta,et al.  Low coherence unit norm tight frames , 2018, ArXiv.

[38]  Vito Cerone,et al.  Direct data-driven control design through set-membership errors-in-variables identification techniques , 2017, 2017 American Control Conference (ACC).

[39]  Mattia Zorzi,et al.  Factor Models With Real Data: A Robust Estimation of the Number of Factors , 2017, IEEE Transactions on Automatic Control.

[40]  Michael Elad,et al.  On the stability of the basis pursuit in the presence of noise , 2006, Signal Process..

[41]  Tyrone L. Vincent,et al.  Compressive System Identification of LTI and LTV ARX models , 2011, IEEE Conference on Decision and Control and European Control Conference.

[42]  Enrico Magli,et al.  GPU-accelerated algorithms for compressed signals recovery with application to astronomical imagery deblurring , 2017, ArXiv.

[43]  Holger Rauhut,et al.  A Mathematical Introduction to Compressive Sensing , 2013, Applied and Numerical Harmonic Analysis.

[44]  Enrico Magli,et al.  Voyager 2 solar plasma and magnetic field spectral analysis for intermediate data sparsity , 2015, 1510.04304.