Algorithms for robust linear regression by exploiting the connection to sparse signal recovery

In this paper, we develop algorithms for robust linear regression by leveraging the connection between the problems of robust regression and sparse signal recovery. We explicitly model the measurement noise as a combination of two terms; the first term accounts for regular measurement noise modeled as zero mean Gaussian noise, and the second term captures the impact of outliers. The fact that the latter outlier component could indeed be a sparse vector provides the opportunity to leverage sparse signal reconstruction methods to solve the problem of robust regression. Maximum a posteriori (MAP) based and empirical Bayesian inference based algorithms are developed for this purpose. Experimental studies on simulated and real data sets are presented to demonstrate the effectiveness of the proposed algorithms.

[1]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[2]  E.J. Candes Compressive Sampling , 2022 .

[3]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[4]  W. Marsden I and J , 2012 .

[5]  D. Cox,et al.  An Analysis of Transformations , 1964 .

[6]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[7]  Bhaskar D. Rao,et al.  Perspectives on Sparse Bayesian Learning , 2003, NIPS.

[8]  Douglas M. Hawkins Identification of Outliers , 1980, Monographs on Applied Probability and Statistics.

[9]  Leo Breiman,et al.  Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author) , 2001, Statistical Science.

[10]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[11]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[12]  F. Hampel Robust estimation: A condensed partial survey , 1973 .

[13]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[14]  David J. C. MacKay,et al.  Comparison of Approximate Methods for Handling Hyperparameters , 1999, Neural Computation.

[15]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[16]  Bhaskar D. Rao,et al.  Sparse Bayesian learning for basis selection , 2004, IEEE Transactions on Signal Processing.

[17]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[18]  Jeremy MG Taylor,et al.  Robust Statistical Modeling Using the t Distribution , 1989 .

[19]  Leo Breiman,et al.  Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author) , 2001 .

[20]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..