Error bounds for maximum likelihood matrix completion under sparse factor models

This paper examines a general class of matrix completion tasks where entry wise observations of the matrix are subject to random noise or corruption. Our particular focus here is on settings where the matrix to be estimated follows a sparse factor model, in the sense that it may be expressed as the product of two matrices, one of which is sparse. We analyze the performance of a sparsity-penalized maximum likelihood approach to such problems to provide a general-purpose estimation result applicable to any of a number of noise/corruption models, and describe its implications in two stylized scenarios - one characterized by additive Gaussian noise, and the other by highly-quantized one-bit observations. We also provide some supporting empirical evidence to validate our theoretical claims in the Gaussian setting.

[1]  V. Koltchinskii,et al.  Nuclear norm penalization and optimal rates for noisy low rank matrix completion , 2010, 1011.6256.

[2]  R. Nowak,et al.  Multiscale likelihood analysis and complexity penalized estimation , 2004, math/0406424.

[3]  Min Xu,et al.  Completion of high-rank ultrametric matrices using selective entries , 2012, 2012 International Conference on Signal Processing and Communications (SPCOM).

[4]  Karin Schnass,et al.  Dictionary Identification—Sparse Matrix-Factorization via $\ell_1$ -Minimization , 2009, IEEE Transactions on Information Theory.

[5]  Huan Wang,et al.  Exact Recovery of Sparse-Used Dictionaries , 2013, IJCAI.

[6]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[7]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[8]  Guillermo Sapiro,et al.  Online dictionary learning for sparse coding , 2009, ICML '09.

[9]  A. Bruckstein,et al.  On the uniqueness of overcomplete dictionaries, and a practical way to retrieve them , 2006 .

[10]  Guillermo Sapiro,et al.  Dictionary Learning for Noisy and Incomplete Hyperspectral Images , 2012, SIAM J. Imaging Sci..

[11]  Ewout van den Berg,et al.  1-Bit Matrix Completion , 2012, ArXiv.

[12]  P. Massart,et al.  Risk bounds for model selection via penalization , 1999 .

[13]  John Wright,et al.  Segmentation of Multivariate Mixed Data via Lossy Data Coding and Compression , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Robert D. Nowak,et al.  High-dimensional Matched Subspace Detection when data are missing , 2010, 2010 IEEE International Symposium on Information Theory.

[15]  Karin Schnass,et al.  On the Identifiability of Overcomplete Dictionaries via the Minimisation Principle Underlying K-SVD , 2013, ArXiv.

[16]  Martin J. Wainwright,et al.  Restricted strong convexity and weighted matrix completion: Optimal bounds with noise , 2010, J. Mach. Learn. Res..

[17]  Robert D. Nowak,et al.  Multiscale Poisson Intensity and Density Estimation , 2007, IEEE Transactions on Information Theory.

[18]  Prateek Jain,et al.  Learning Sparsely Used Overcomplete Dictionaries via Alternating Minimization , 2013, SIAM J. Optim..

[19]  Huan Wang,et al.  On the local correctness of ℓ1-minimization for dictionary learning , 2011, 2014 IEEE International Symposium on Information Theory.

[20]  Shiqian Ma,et al.  Fixed point and Bregman iterative methods for matrix rank minimization , 2009, Math. Program..

[21]  Jarvis D. Haupt,et al.  Estimation error guarantees for Poisson denoising with sparse and structured dictionary models , 2014, 2014 IEEE International Symposium on Information Theory.

[22]  Robert D. Nowak,et al.  High-Rank Matrix Completion and Subspace Clustering with Missing Data , 2011, ArXiv.

[23]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[24]  A. Barron,et al.  Estimation of mixture models , 1999 .

[25]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[26]  Roummel F. Marcia,et al.  Compressed Sensing Performance Bounds Under Poisson Noise , 2009, IEEE Transactions on Signal Processing.

[27]  Robert D. Nowak,et al.  K-subspaces with missing data , 2012, 2012 IEEE Statistical Signal Processing Workshop (SSP).

[28]  Rémi Gribonval,et al.  Local stability and robustness of sparse dictionary learning in the presence of noise , 2012, ArXiv.

[29]  Nikos D. Sidiropoulos,et al.  Sparse dictionary learning from 1-BIT data , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[30]  Jarvis D. Haupt,et al.  Noisy Matrix Completion Under Sparse Factor Models , 2014, IEEE Transactions on Information Theory.

[31]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[32]  Olgica Milenkovic,et al.  SET: An algorithm for consistent matrix completion , 2009, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[33]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, ISIT.

[34]  Huan Wang,et al.  Exact Recovery of Sparsely-Used Dictionaries , 2012, COLT.

[35]  P. Tseng.  Nearest q-Flat to m Points , 2000 .

[36]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[37]  R. Tibshirani,et al.  Sparse Principal Component Analysis , 2006 .

[38]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[39]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.

[40]  David B. Dunson,et al.  Nonparametric Bayesian Dictionary Learning for Analysis of Noisy and Incomplete Images , 2012, IEEE Transactions on Image Processing.