Graph Signal Sampling for Inductive One-Bit Matrix Completion: a Closed-form Solution

Inductive one-bit matrix completion is motivated by modern applications such as recommender systems, where new users would appear at test stage with the ratings consisting of only ones and no zeros. We propose a unified graph signal sampling framework which enjoys the benefits of graph signal analysis and processing. The key idea is to transform each user's ratings on the items to a function (signal) on the vertices of an item-item graph, then learn structural graph properties to recover the function from its values on certain vertices -- the problem of graph signal sampling. We propose a class of regularization functionals that takes into account discrete random label noise in the graph vertex domain, then develop the GS-IMC approach which biases the reconstruction towards functions that vary little between adjacent vertices for noise reduction. Theoretical result shows that accurate reconstructions can be achieved under mild conditions. For the online setting, we develop a Bayesian extension, i.e., BGS-IMC which considers continuous random Gaussian noise in the graph Fourier domain and builds upon a prediction-correction update algorithm to obtain the unbiased and minimum-variance reconstruction. Both GS-IMC and BGS-IMC have closed-form solutions and thus are highly scalable in large data. Experiments show that our methods achieve state-of-the-art performance on public benchmarks.

[1]  Lorenzo Livi,et al.  Graph Neural Networks With Convolutional ARMA Filters , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Petko Bogdanov,et al.  Temporal Graph Signal Decomposition , 2021, KDD.

[3]  Junchi Yan,et al.  Scalable and Explainable 1-Bit Matrix Completion via Graph Signal Learning , 2021, AAAI.

[4]  Junchi Yan,et al.  Towards Open-World Recommendation: An Inductive Model-based Collaborative Filtering Approach , 2020, ICML.

[5]  M. Kloft,et al.  Fine-grained Generalization Analysis of Inductive Matrix Completion , 2021, NeurIPS.

[6]  Xiangnan He,et al.  LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation , 2020, SIGIR.

[7]  G. Guo,et al.  Future Data Helps Training: Modeling Future Contexts for Session-based Recommendation , 2019, WWW.

[8]  Yixin Chen,et al.  Inductive Matrix Completion Based on Graph Neural Networks , 2019, ICLR.

[9]  Patrick Gallinari,et al.  Normalizing Kalman Filters for Multivariate Time Series Analysis , 2020, NeurIPS.

[10]  Harald Steck,et al.  Markov Random Fields for Collaborative Filtering , 2019, NeurIPS.

[11]  Tat-Seng Chua,et al.  Neural Graph Collaborative Filtering , 2019, SIGIR.

[12]  Peng Jiang,et al.  BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer , 2019, CIKM.

[13]  U. Feige,et al.  Spectral Graph Theory , 2015 .

[14]  Kilian Q. Weinberger,et al.  Simplifying Graph Convolutional Networks , 2019, ICML.

[15]  Georgios B. Giannakis,et al.  Matrix Completion and Extrapolation via Kernel Regression , 2018, IEEE Transactions on Signal Processing.

[16]  Inderjit S. Dhillon,et al.  Provable Non-linear Inductive Matrix Completion , 2019, NeurIPS.

[17]  Yuantao Gu,et al.  Spatio-Temporal Signal Recovery Based on Low Rank and Differential Smoothness , 2018, IEEE Transactions on Signal Processing.

[18]  Julian J. McAuley,et al.  Self-Attentive Sequential Recommendation , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[19]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[20]  Matthew D. Hoffman,et al.  Variational Autoencoders for Collaborative Filtering , 2018, WWW.

[21]  Enrico Magli,et al.  Graph Spectral Image Processing , 2018, Proceedings of the IEEE.

[22]  Pierre Vandergheynst,et al.  Graph Signal Processing: Overview, Challenges, and Applications , 2017, Proceedings of the IEEE.

[23]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[24]  Inderjit S. Dhillon,et al.  Using Side Information to Reliably Learn Low-Rank Matrices from Missing and Corrupted Observations , 2018, J. Mach. Learn. Res..

[25]  Cho-Jui Hsieh,et al.  Large-scale Collaborative Ranking in Near-Linear Time , 2017, KDD.

[26]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[27]  Max Welling,et al.  Graph Convolutional Matrix Completion , 2017, ArXiv.

[28]  Alex Beutel,et al.  Recurrent Recommender Networks , 2017, WSDM.

[29]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[30]  Georgios B. Giannakis,et al.  Kernel-Based Reconstruction of Graph Signals , 2016, IEEE Transactions on Signal Processing.

[31]  Gene Cheung,et al.  Graph Laplacian Regularization for Image Denoising: Analysis in the Continuous Domain , 2016, IEEE Transactions on Image Processing.

[32]  Yu-Xiang Wang,et al.  Higher-Order Total Variation Classes on Grids: Minimax Theory and Trend Filtering Methods , 2017, NIPS.

[33]  Paul Voigt,et al.  The EU General Data Protection Regulation (GDPR) , 2017 .

[34]  Julian J. McAuley,et al.  Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[35]  Tat-Seng Chua,et al.  Fast Matrix Factorization for Online Recommendation with Implicit Feedback , 2016, SIGIR.

[36]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[37]  Yu-Xiang Wang,et al.  Total Variation Classes Beyond 1d: Minimax Rates, and the Limitations of Linear Smoothers , 2016, NIPS.

[38]  Xavier Bresson,et al.  Song recommendation with non-negative matrix factorization and graph total variation , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[39]  Antonio Ortega,et al.  Submitted to Ieee Transactions on Signal Processing 1 Efficient Sampling Set Selection for Bandlimited Graph Signals Using Graph Spectral Proxies , 2022 .

[40]  Aryan Mokhtari,et al.  A Class of Prediction-Correction Methods for Time-Varying Convex Optimization , 2015, IEEE Transactions on Signal Processing.

[41]  Maksims Volkovs,et al.  Effective Latent Models for Binary Feedback in Recommender Systems , 2015, SIGIR.

[42]  Nicolas Kourtellis,et al.  Dynamic Matrix Factorization with Priors on Unknown Values , 2015, KDD.

[43]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[44]  Miao Xu,et al.  Speedup Matrix Completion with Side Information: Application to Multi-Label Learning , 2013, NIPS.

[45]  Inderjit S. Dhillon,et al.  Provable Inductive Matrix Completion , 2013, ArXiv.

[46]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[47]  Peter S. Maybeck,et al.  Stochastic Models, Estimation And Control , 2012 .

[48]  Jian Huang,et al.  The Sparse Laplacian Shrinkage Estimator for High-Dimensional Regression. , 2011, Annals of statistics.

[49]  Isaac Z. Pesenson,et al.  Variational Splines and Paley–Wiener Spaces on Combinatorial Graphs , 2009, ArXiv.

[50]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[51]  I. Pesenson Sampling in paley-wiener spaces on combinatorial graphs , 2008, 1111.5896.

[52]  José M. F. Moura,et al.  Distributing the Kalman Filter for Large-Scale Systems , 2007, IEEE Transactions on Signal Processing.

[53]  James Bennett,et al.  The Netflix Prize , 2007 .

[54]  Bernhard Schölkopf,et al.  Learning with Hypergraphs: Clustering, Classification, and Embedding , 2006, NIPS.

[55]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[56]  Mikhail Belkin,et al.  Regularization and Semi-supervised Learning on Large Graphs , 2004, COLT.

[57]  Larry H. Matthies,et al.  Kalman filter-based algorithms for estimating depth from image sequences , 1989, International Journal of Computer Vision.

[58]  Jitendra Malik,et al.  Spectral grouping using the Nystrom method , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[59]  Alexander J. Smola,et al.  Kernels and Regularization on Graphs , 2003, COLT.

[60]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[61]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[62]  Isaac Z. Pesenson,et al.  A sampling theorem on homogeneous manifolds , 2000 .

[63]  S. Varadhan,et al.  Diffusion processes with continuous coefficients, I , 1969 .

[64]  Norbert Wiener,et al.  Extrapolation, Interpolation, and Smoothing of Stationary Time Series, with Engineering Applications , 1949 .

[65]  G. R. Maclane Concerning the uniformization of certain Riemann surfaces allied to the inverse-cosine and inverse-gamma surfaces , 1947 .

[66]  Rayleigh The Problem of the Random Walk , 1905, Nature.