Noisy inductive matrix completion under sparse factor models

Inductive Matrix Completion (IMC) is an important class of matrix completion problems that allows direct inclusion of available features to enhance estimation capabilities. These models have found applications in personalized recommendation systems, multilabel learning, dictionary learning, etc. This paper examines a general class of noisy matrix completion tasks where the underlying matrix is following an IMC model i.e., it is formed by a mixing matrix (a priori unknown) sandwiched between two known feature matrices. The mixing matrix here is assumed to be well approximated by the product of two sparse matrices — referred here to as “sparse factor models.” We leverage the main theorem of [1] and extend it to provide theoretical error bounds for the sparsity-regularized maximum likelihood estimators for the class of problems discussed in this paper. The main result is general in the sense that it can be used to derive error bounds for various noise models. In this paper, we instantiate our main result for the case of Gaussian noise and provide corresponding error bounds in terms of squared loss.

[1]  A. Barron,et al.  THE MDL PRINCIPLE , PENALIZED LIKELIHOODS , AND STATISTICAL RISK , 2008 .

[2]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[3]  Jarvis D. Haupt,et al.  Error bounds for maximum likelihood matrix completion under sparse factor models , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[4]  Sanjeev Arora,et al.  Computing a nonnegative matrix factorization -- provably , 2011, STOC '12.

[5]  Inderjit S. Dhillon,et al.  Tumblr Blog Recommendation with Boosted Inductive Matrix Completion , 2015, CIKM.

[6]  Ewout van den Berg,et al.  1-Bit Matrix Completion , 2012, ArXiv.

[7]  Inderjit S. Dhillon,et al.  Matrix Completion with Noisy Side Information , 2015, NIPS.

[8]  Inderjit S. Dhillon,et al.  Provable Inductive Matrix Completion , 2013, ArXiv.

[9]  Andrea Montanari,et al.  Matrix Completion from Noisy Entries , 2009, J. Mach. Learn. Res..

[10]  Rasmus Bro,et al.  Coupled Matrix Factorization with Sparse Factors to Identify Potential Biomarkers in Metabolomics , 2012, 2012 IEEE 12th International Conference on Data Mining Workshops.

[11]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[12]  Huan Wang,et al.  Exact Recovery of Sparsely-Used Dictionaries , 2012, COLT.

[13]  Inderjit S. Dhillon,et al.  Goal-Directed Inductive Matrix Completion , 2016, KDD.

[14]  A. Bruckstein,et al.  On the uniqueness of overcomplete dictionaries, and a practical way to retrieve them , 2006 .

[15]  Martin J. Wainwright,et al.  Restricted strong convexity and weighted matrix completion: Optimal bounds with noise , 2010, J. Mach. Learn. Res..

[16]  Patrick Seemann,et al.  Matrix Factorization Techniques for Recommender Systems , 2014 .

[17]  Jarvis D. Haupt,et al.  Estimation error guarantees for Poisson denoising with sparse and structured dictionary models , 2014, 2014 IEEE International Symposium on Information Theory.

[18]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[19]  Steffen Rendle,et al.  Factorization Machines , 2010, 2010 IEEE International Conference on Data Mining.

[20]  Michael Möller,et al.  A Convex Model for Nonnegative Matrix Factorization and Dimensionality Reduction on Physical Space , 2011, IEEE Transactions on Image Processing.

[21]  Max Welling,et al.  Bayesian Matrix Factorization with Side Information and Dirichlet Process Mixtures , 2010, AAAI.

[22]  Jarvis D. Haupt,et al.  Noisy Matrix Completion Under Sparse Factor Models , 2014, IEEE Transactions on Information Theory.

[23]  Miao Xu,et al.  Speedup Matrix Completion with Side Information: Application to Multi-Label Learning , 2013, NIPS.

[24]  Michael Elad,et al.  Double Sparsity: Learning Sparse Dictionaries for Sparse Signal Approximation , 2010, IEEE Transactions on Signal Processing.

[25]  Michael R. Lyu,et al.  SoRec: social recommendation using probabilistic matrix factorization , 2008, CIKM '08.

[26]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[27]  Jarvis D. Haupt,et al.  On the Fundamental Limits of Recovering Tree Sparse Vectors From Noisy Linear Measurements , 2013, IEEE Transactions on Information Theory.

[28]  Nikos D. Sidiropoulos,et al.  Sparse dictionary learning from 1-BIT data , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[29]  Karin Schnass,et al.  Dictionary Identification—Sparse Matrix-Factorization via $\ell_1$ -Minimization , 2009, IEEE Transactions on Information Theory.

[30]  Deepak Agarwal,et al.  fLDA: matrix factorization through latent dirichlet allocation , 2010, WSDM '10.