Computational Barriers in Minimax Submatrix Detection

This paper studies the minimax detection of a small submatrix of elevated mean in a large matrix contaminated by additive Gaussian noise. To investigate the tradeoff between statistical performance and computational cost from a complexity-theoretic perspective, we consider a sequence of discretized models which are asymptotically equivalent to the Gaussian model. Under the hypothesis that the planted clique detection problem cannot be solved in randomized polynomial time when the clique size is of smaller order than the square root of the graph size, the following phase transition phenomenon is established: when the size of the large matrix $p\to\infty$, if the submatrix size $k=\Theta(p^{\alpha})$ for any $\alpha\in(0,{2}/{3})$, computational complexity constraints can incur a severe penalty on the statistical performance in the sense that any randomized polynomial-time test is minimax suboptimal by a polynomial factor in $p$; if $k=\Theta(p^{\alpha})$ for any $\alpha\in({2}/{3},1)$, minimax optimal detection can be attained within constant factors in linear time. Using Schatten norm loss as a representative example, we show that the hardness of attaining the minimax estimation rate can crucially depend on the loss function. Implications on the hardness of support recovery are also obtained.

[1]  Ludek Kucera,et al.  A Generalized Encryption Scheme Based on Random Graphs , 1991, WG.

[2]  T. Koshy Catalan Numbers with Applications , 2008 .

[3]  A. Nobel,et al.  Finding large average submatrices in high dimensional data , 2009, 0905.1682.

[4]  Philippe Rigollet,et al.  Complexity Theoretic Lower Bounds for Sparse Principal Component Detection , 2013, COLT.

[5]  Yihong Wu,et al.  Volume ratio, sparsity, and minimaxity under unitarily invariant norms , 2013, 2013 IEEE International Symposium on Information Theory.

[6]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[7]  Michael I. Jordan,et al.  Computational and statistical tradeoffs via convex relaxation , 2012, Proceedings of the National Academy of Sciences.

[8]  Yuval Peres,et al.  Finding Hidden Cliques in Linear Time with High Probability , 2010, Combinatorics, Probability and Computing.

[9]  A. Nobel,et al.  On the maximal size of large-average and ANOVA-fit submatrices in a Gaussian random matrix. , 2010, Bernoulli : official journal of the Bernoulli Society for Mathematical Statistics and Probability.

[10]  Sivaraman Balakrishnan,et al.  Minimax Localization of Structural Information in Large Noisy Matrices , 2011, NIPS.

[11]  A. Rényi On the dimension and entropy of probability distributions , 1959 .

[12]  U. Feige,et al.  Finding hidden cliques in linear time , 2009 .

[13]  Thomas M. Cover,et al.  Elements of information theory (2. ed.) , 2006 .

[14]  A. Nobel,et al.  Energy landscape for large average submatrix detection problems in Gaussian random matrices , 2012, 1211.2284.

[15]  B. Nadler,et al.  DO SEMIDEFINITE RELAXATIONS SOLVE SPARSE PCA UP TO THE INFORMATION LIMIT , 2013, 1306.3690.

[16]  李幼升,et al.  Ph , 1989 .

[17]  Noga Alon,et al.  Testing k-wise and almost k-wise independence , 2007, STOC '07.

[18]  Yu. I. Ingster,et al.  Detection of a sparse submatrix of a high-dimensional noisy matrix , 2011, 1109.0898.

[19]  Andrea Montanari,et al.  Finding Hidden Cliques of Size $$\sqrt{N/e}$$N/e in Nearly Linear Time , 2013, Found. Comput. Math..

[20]  Ludek Kucera,et al.  Expected Complexity of Graph Partitioning Problems , 1995, Discret. Appl. Math..

[21]  Noga Alon,et al.  Finding a large hidden clique in a random graph , 1998, SODA '98.

[22]  E. Arias-Castro,et al.  Community Detection in Sparse Random Networks , 2013, 1308.2955.

[23]  Pascal Koiran,et al.  Hidden Cliques and the Certification of the Restricted Isometry Property , 2012, IEEE Transactions on Information Theory.

[24]  D. A. Edwards On the existence of probability measures with given marginals , 1978 .

[25]  Sanjeev Arora,et al.  Computational Complexity: A Modern Approach , 2009 .

[26]  L. Wasserman,et al.  Statistical and computational tradeoffs in biclustering , 2011 .

[27]  Stephen A. Vavasis,et al.  Nuclear norm minimization for the planted clique and biclique problems , 2009, Math. Program..

[28]  A. Shiryaev,et al.  Statistical Experiments and Decisions: Asymptotic Theory , 2000 .

[29]  Mark Jerrum,et al.  Large Cliques Elude the Metropolis Process , 1992, Random Struct. Algorithms.

[30]  Avi Wigderson,et al.  Public-key cryptography from different assumptions , 2010, STOC '10.

[31]  Robert Krauthgamer,et al.  How hard is it to approximate the best Nash equilibrium? , 2009, SODA.

[32]  Robert Krauthgamer,et al.  Finding and certifying a large hidden clique in a semirandom graph , 2000, Random Struct. Algorithms.

[33]  Luc Devroye,et al.  Combinatorial Testing Problems , 2009, 0908.3437.

[34]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[35]  L. L. Cam,et al.  Asymptotic Methods In Statistical Decision Theory , 1986 .

[36]  E. Arias-Castro,et al.  Community Detection in Random Networks , 2013, 1302.7099.

[37]  Ari Juels,et al.  Hiding Cliques for Cryptographic Security , 1998, SODA '98.

[38]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[39]  P. Rigollet,et al.  Optimal detection of sparse principal components in high dimension , 2012, 1202.5070.

[40]  B. Nadler,et al.  Do Semidefinite Relaxations Really Solve Sparse PCA , 2013 .

[41]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[42]  Benjamin Rossman,et al.  Average-case complexity of detecting cliques , 2010 .

[43]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..