Adaptive sensing performance lower bounds for sparse estimation and testing

This paper gives a precise characterization of the fundamental limits of adaptive sensing for diverse estimation and testing problems concerning sparse signals. We consider in particular the setting introduced in Haupt, Castro and Nowak (2011) and show necessary conditions on the minimum signal magnitude for both detection and estimation: if $x\in\R^n$ is a sparse vector with $s$ non-zero components then it can be reliably detected in noise provided the magnitude of the non-zero components exceeds $\sqrt{2/s}$. Furthermore, the signal support can be exactly identified provided the minimum magnitude exceeds $\sqrt{2\log s}$. Notably there is no dependence on $n$, the extrinsic signal dimension. These results show that the adaptive sensing methodologies proposed previously in the literature are essentially optimal, and cannot be substantially improved. In addition these results provide further insights on the limits of adaptive compressive sensing.

[1]  Emmanuel J. Candès,et al.  On the Fundamental Limits of Adaptive Sensing , 2011, IEEE Transactions on Information Theory.

[2]  Robert D. Nowak,et al.  Sequentially designed compressed sensing , 2012, 2012 IEEE Statistical Signal Processing Workshop (SSP).

[3]  Matthew Malloy,et al.  On the limits of sequential testing in high dimensions , 2011, 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[4]  Matthew Malloy,et al.  Sequential analysis in high-dimensional multiple testing and sparse recovery , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[5]  Steve Hanneke.  Rates of convergence in active learning , 2011, 1103.1790.

[6]  Robert D. Nowak,et al.  Distilled Sensing: Adaptive Sampling for Sparse Detection and Estimation , 2010, IEEE Transactions on Information Theory.

[7]  Vladimir Koltchinskii,et al.  Rademacher Complexities and Bounding the Excess Risk in Active Learning , 2010, J. Mach. Learn. Res..

[8]  Luc Devroye,et al.  Combinatorial Testing Problems , 2009, 0908.3437.

[9]  Alexandre B. Tsybakov,et al.  Introduction to Nonparametric Estimation , 2008, Springer series in statistics.

[10]  Robert D. Nowak,et al.  Minimax Bounds for Active Learning , 2007, IEEE Transactions on Information Theory.

[11]  E. Candès,et al.  Searching for a trail of evidence in a maze , 2007, math/0701668.

[12]  N. Meinshausen,et al.  Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses , 2005, math/0501289.

[13]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[14]  Adam Tauman Kalai,et al.  Analysis of Perceptron-Based Active Learning , 2009, COLT.

[15]  Sanjoy Dasgupta,et al.  Coarse sample complexity bounds for active learning , 2005, NIPS.

[16]  Robert D. Nowak,et al.  Faster Rates in Regression via Active Learning , 2005, NIPS.

[17]  L. Wasserman.  All of Nonparametric Statistics , 2005 .

[18]  D. Geman,et al.  Hierarchical testing designs for pattern recognition , 2005, math/0507421.

[19]  Sanjoy Dasgupta,et al.  Analysis of a greedy active learning strategy , 2004, NIPS.

[20]  D. Donoho,et al.  Higher criticism for detecting sparse heterogeneous mixtures , 2004, math/0410072.

[21]  H. Sebastian Seung,et al.  Selective Sampling Using the Query by Committee Algorithm , 1997, Machine Learning.

[22]  P. Hall,et al.  Sequential methods for design-adaptive estimation of discontinuities in regression curves and surfaces , 2003 .

[23]  A. Tsybakov,et al.  Introduction à l'estimation non-paramétrique , 2003 .

[24]  Yu. I. Ingster,et al.  Nonparametric Goodness-of-Fit Testing Under Gaussian Models , 2002 .

[25]  A. Korostelev,et al.  Rates of convergence for the sup-norm risk in image models under sequential designs , 2000 .

[26]  Erich Novak,et al.  On the Power of Adaption , 1996, J. Complex..

[27]  David A. Cohn,et al.  Active Learning with Statistical Models , 1996, NIPS.

[28]  Mahmoud A. El-Gamal.  The Role of Priors in Active Bayesian Learning in the Sequential Statistical Decision Framework , 1991 .

[29]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[30]  H. Chernoff.  Sequential Design of Experiments , 1959 .