Learning Discrete Matrix Factorization Models

Matrix factorization is among the most popular approaches for matrix completion, with recent advances including gradient-based and deep-learning-based methods. Even though many applications involve matrices with discrete values, most of the existing matrix factorization models focus on the continuous domain. Discretization is applied as an additional step, often using a heuristic mapping that results in sub-optimal solutions, which either do not take into account the structure of the matrix or introduce significant quantization errors. In this letter, we propose a novel method that allows gradient-based and deep-learning-based methods to jointly learn both the matrix factorization model and a discretization operator. By introducing a loss function that accounts for the reconstruction error with respect to the discrete predictions, we obtain a discrete matrix completion algorithm with high reconstruction accuracy. Experiments using well-known datasets show the improvement obtained by the proposed algorithm over the state of the art.

[1]  Yonina C. Eldar,et al.  Phase Retrieval via Matrix Completion , 2011, SIAM Rev..

[2]  Feiping Nie,et al.  Robust Manifold Nonnegative Matrix Factorization , 2014, ACM Trans. Knowl. Discov. Data.

[3]  Shujian Huang,et al.  Deep Matrix Factorization Models for Recommender Systems , 2017, IJCAI.

[4]  F. Maxwell Harper,et al.  The MovieLens Datasets: History and Context , 2016, TIIS.

[5]  Gonzalo Mateos,et al.  Modeling and Optimization for Big Data Analytics: (Statistical) learning tools for our era of data deluge , 2014, IEEE Signal Processing Magazine.

[6]  Nitish Srivastava,et al.  Unsupervised Learning of Video Representations using LSTMs , 2015, ICML.

[7]  Eugene L. Allgower,et al.  Numerical continuation methods - an introduction , 1990, Springer series in computational mathematics.

[8]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[9]  E. Allgower,et al.  Numerical Continuation Methods , 1990 .

[10]  Feiping Nie,et al.  Robust Discrete Matrix Completion , 2013, AAAI.

[11]  Yee Whye Teh,et al.  The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables , 2016, ICLR.

[12]  Richard G. Baraniuk,et al.  A deep learning approach to structured signal recovery , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[13]  Duc Minh Nguyen,et al.  Deep learning sparse ternary projections for compressed sensing of images , 2017, 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[14]  Heng Huang,et al.  Optimal Discrete Matrix Completion , 2016, AAAI.

[15]  Yong Luo,et al.  Multiview matrix completion for multilabel image classification. , 2015, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[16]  Guangdong Feng,et al.  A Tensor Based Method for Missing Traffic Data Completion , 2013 .

[17]  Feiping Nie,et al.  Joint Capped Norms Minimization for Robust Matrix Recovery , 2017, IJCAI.

[18]  Geoffrey J. Gordon,et al.  A Unified View of Matrix Factorization Models , 2008, ECML/PKDD.

[19]  Xinwei Yang,et al.  Effective exemplar-based image inpainting using low-rank matrix completion , 2015, 2015 IEEE 7th International Conference on Awareness Science and Technology (iCAST).

[20]  David L. Neuhoff,et al.  Quantization , 2022, IEEE Trans. Inf. Theory.

[21]  Seungjin Choi,et al.  Semi-Supervised Nonnegative Matrix Factorization , 2010, IEEE Signal Processing Letters.

[22]  Xuelong Li,et al.  Robust Alternative Minimization for Matrix Completion , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[23]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[24]  Feiping Nie,et al.  Joint Schatten $$p$$p-norm and $$\ell _p$$ℓp-norm robust matrix completion for missing value recovery , 2013, Knowledge and Information Systems.

[25]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, ISIT.

[26]  Luo Si,et al.  Preference preserving hashing for efficient recommendation , 2014, SIGIR.

[27]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Patrick Seemann,et al.  Matrix Factorization Techniques for Recommender Systems , 2014 .

[29]  Philip S. Yu,et al.  HashNet: Deep Learning to Hash by Continuation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[30]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[31]  Hongyuan Zha,et al.  Learning binary codes for collaborative filtering , 2012, KDD.

[32]  Sonia A. Bhaskar,et al.  Probabilistic Low-Rank Matrix Completion from Quantized Measurements , 2016, J. Mach. Learn. Res..

[33]  Duc Minh Nguyen,et al.  Extendable Neural Matrix Completion , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[34]  Feiping Nie,et al.  Social Trust Prediction Using Rank-k Matrix Recovery , 2013, IJCAI.

[35]  Sean M. McNee,et al.  Improving recommendation lists through topic diversification , 2005, WWW '05.

[36]  Sewoong Oh,et al.  A Gradient Descent Algorithm on the Grassman Manifold for Matrix Completion , 2009, ArXiv.

[37]  Ben Poole,et al.  Categorical Reparameterization with Gumbel-Softmax , 2016, ICLR.

[38]  Daniel M. Roy,et al.  Neural Network Matrix Factorization , 2015, ArXiv.

[39]  Richard G. Baraniuk,et al.  Matrix recovery from quantized and corrupted measurements , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[40]  Bin Shen,et al.  Semisupervised Nonnegative Matrix Factorization for learning the semantics , 2012, The 6th International Conference on Soft Computing and Intelligent Systems, and The 13th International Symposium on Advanced Intelligence Systems.

[41]  Feiping Nie,et al.  Low-Rank Matrix Recovery via Efficient Schatten p-Norm Minimization , 2012, AAAI.

[42]  Alexandre Bernardino,et al.  Unifying Nuclear Norm and Bilinear Factorization Approaches for Low-Rank Matrix Decomposition , 2013, 2013 IEEE International Conference on Computer Vision.

[43]  Ole Winther,et al.  A hierarchical model for ordinal matrix factorization , 2012, Stat. Comput..

[44]  Jinfeng Yi,et al.  Robust Ensemble Clustering by Matrix Completion , 2012, 2012 IEEE 12th International Conference on Data Mining.