A Minimax Lower Bound for Low-Rank Matrix-Variate Logistic Regression

This paper considers the problem of matrix-variate logistic regression. The fundamental error threshold on estimating coefficient matrices in the logistic regression problem is found by deriving a lower bound on the minimax risk. The focus of this paper is on derivation of a minimax risk lower bound for low-rank coefficient matrices. The bound depends explicitly on the dimensions and distribution of the covariates, the rank and energy of the coefficient matrix, and the number of samples. The resulting bound is proportional to the intrinsic degrees of freedom in the problem, which suggests the sample complexity of the lowrank matrix logistic regression problem can be lower than that for vectorized logistic regression. The proof techniques utilized in this work also set the stage for development of minimax lower bounds for tensor-variate logistic regression problems.

[1]  Mark C Pierce,et al.  Computational endoscopy-a framework for improving spatial resolution in fiber bundle imaging. , 2019, Optics letters.

[2]  Yonina C. Eldar,et al.  On the Minimax Risk of Dictionary Learning , 2015, IEEE Transactions on Information Theory.

[3]  Anand D. Sarwate,et al.  Minimax Lower Bounds on Dictionary Learning for Tensor Data , 2016, IEEE Transactions on Information Theory.

[4]  Haipeng Luo,et al.  Logistic Regression: The Importance of Being Improper , 2018, COLT.

[5]  Hung Hung,et al.  Matrix variate logistic regression model with application to EEG data. , 2011, Biostatistics.

[6]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[7]  Beibei Zhang,et al.  Logistic regression with image covariates via the combination of L1 and Sobolev regularizations , 2020, PloS one.

[8]  Martin J. Wainwright,et al.  Information-Theoretic Limits on Sparsity Recovery in the High-Dimensional and Noisy Setting , 2007, IEEE Transactions on Information Theory.

[9]  Anru Zhang,et al.  ISLET: Fast and Optimal Low-rank Tensor Regression via Importance Sketching , 2020, SIAM J. Math. Data Sci..

[10]  Kristin L. Sainani,et al.  Logistic Regression , 2014, PM & R : the journal of injury, function, and rehabilitation.

[11]  Jianmin Jiang,et al.  Rank-Optimized Logistic Matrix Regression toward Improved Matrix Data Classification , 2018, Neural Computation.

[12]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[13]  Alessandro Panconesi,et al.  Concentration of Measure for the Analysis of Randomized Algorithms , 2009 .

[14]  Anand D. Sarwate,et al.  Minimax lower bounds for Kronecker-structured dictionary learning , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[15]  Martin J. Wainwright,et al.  Restricted strong convexity and weighted matrix completion: Optimal bounds with noise , 2010, J. Mach. Learn. Res..

[16]  Quentin Berthet,et al.  Statistical and Computational Rates in Graph Logistic Regression , 2020, AISTATS.

[17]  Bin Yu.  Assouad, Fano, and Le Cam , 1997 .

[18]  Garvesh Raskutti,et al.  Convex regularization for high-dimensional multiresponse tensor regression , 2015, The Annals of Statistics.

[19]  R. Khasminskii.  A Lower Bound on the Risks of Non-Parametric Estimates of Densities in the Uniform Metric , 1979 .

[20]  Ayfer Özgür,et al.  Minimax Bounds for Distributed Logistic Regression , 2019, ArXiv.

[21]  Waheed U. Bajwa,et al.  Learning Product Graphs Underlying Smooth Graph Signals , 2020, ArXiv.

[22]  Felix Abramovich,et al.  High-Dimensional Classification by Sparse Logistic Regression , 2017, IEEE Transactions on Information Theory.

[23]  Richard G. Baraniuk,et al.  Sparse Bilinear Logistic Regression , 2014, ArXiv.