On the theoretical analysis of cross validation in compressive sensing

Compressive sensing (CS) is a data acquisition technique that measures sparse or compressible signals at a sampling rate lower than their Nyquist rate. Results show that sparse signals can be reconstructed using greedy algorithms, often requiring prior knowledge such as the signal sparsity or the noise level. As a substitute to prior knowledge, cross validation (CV), a statistical method that examines whether a model overfits its data, has been proposed to determine the stopping condition of greedy algorithms. This paper analyses cross validation in a general compressive sensing framework. Furthermore, we provide both theoretical analysis and numerical simulations for a cross-validation modification of orthogonal matching pursuit, referred to as OMP-CV, which has good performance in sparse recovery.

[1]  R. Dennis Cook,et al.  Cross-Validation of Regression Models , 1984 .

[2]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[3]  E.J. Candes Compressive Sampling , 2022 .

[4]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[5]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[6]  Yuantao Gu,et al.  Cross Validation in Compressive Sensing and its Application of OMP-CV Algorithm , 2016, ArXiv.

[7]  Richard G. Baraniuk,et al.  Sparse Signal Reconstruction from Noisy Compressive Measurements using Cross Validation , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.

[8]  Rachel Ward,et al.  Compressed Sensing With Cross Validation , 2008, IEEE Transactions on Information Theory.

[9]  E. Candès,et al.  Error correction via linear programming , 2005, FOCS 2005.

[10]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[11]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[12]  Deanna Needell,et al.  Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit , 2007, Found. Comput. Math..

[13]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[14]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing Signal Reconstruction , 2008, IEEE Transactions on Information Theory.

[15]  J. Shao Linear Model Selection by Cross-validation , 1993 .

[16]  Jean-Luc Starck,et al.  Sparse Solution of Underdetermined Systems of Linear Equations by Stagewise Orthogonal Matching Pursuit , 2012, IEEE Transactions on Information Theory.

[17]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.