Sparse Group Inductive Matrix Completion

We consider the problem of matrix completion with side information (\textit{inductive matrix completion}). In real-world applications many side-channel features are typically non-informative making feature selection an important part of the problem. We incorporate feature selection into inductive matrix completion by proposing a matrix factorization framework with group-lasso regularization on side feature parameter matrices. We demonstrate, that the theoretical sample complexity for the proposed method is much lower compared to its competitors in sparse problems, and propose an efficient optimization algorithm for the resulting low-rank matrix completion problem with sparsifying regularizers. Experiments on synthetic and real-world datasets show that the proposed approach outperforms other methods.

[1]  Swayambhoo Jain,et al.  Noisy inductive matrix completion under sparse factor models , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[2]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[3]  Yuhong Guo,et al.  Convex Co-Embedding for Matrix Completion with Predictive Side Information , 2017, AAAI.

[4]  Massimiliano Pontil,et al.  Structured Sparsity and Generalization , 2011, J. Mach. Learn. Res..

[5]  Nathan Srebro,et al.  Fast maximum margin matrix factorization for collaborative prediction , 2005, ICML.

[6]  Inderjit S. Dhillon,et al.  Goal-Directed Inductive Matrix Completion , 2016, KDD.

[7]  Pascal Bianchi,et al.  A primal-dual algorithm for distributed optimization , 2014, 53rd IEEE Conference on Decision and Control.

[8]  Massimiliano Pontil,et al.  Convex multi-task feature learning , 2008, Machine Learning.

[9]  Chih-Jen Lin,et al.  Trust Region Newton Method for Logistic Regression , 2008, J. Mach. Learn. Res..

[10]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[11]  Xiao Zhang,et al.  Fast and Sample Efficient Inductive Matrix Completion via Multi-Phase Procrustes Flow , 2018, ICML.

[12]  James G. Scott,et al.  Proximal Algorithms in Statistics and Machine Learning , 2015, ArXiv.

[13]  John Hannon,et al.  Recommending twitter users to follow using content and collaborative filtering approaches , 2010, RecSys '10.

[14]  José M. Bioucas-Dias,et al.  Alternating direction algorithms for constrained sparse regression: Application to hyperspectral unmixing , 2010, 2010 2nd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing.

[15]  Thomas R Rogers,et al.  Rapid, comprehensive, and affordable mycobacterial diagnosis with whole-genome sequencing: a prospective study , 2016, The Lancet. Respiratory medicine.

[16]  Qing Ling,et al.  Decentralized Dynamic Optimization Through the Alternating Direction Method of Multipliers , 2013, IEEE Transactions on Signal Processing.

[17]  David Suter,et al.  Recovering the missing components in a large noisy low-rank matrix: application to SFM , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Miao Xu,et al.  Speedup Matrix Completion with Side Information: Application to Multi-Label Learning , 2013, NIPS.

[19]  Jinbo Bi,et al.  A Sparse Interactive Model for Matrix Completion with Side Information , 2016, NIPS.

[20]  Peter L. Bartlett,et al.  Rademacher and Gaussian Complexities: Risk Bounds and Structural Results , 2003, J. Mach. Learn. Res..

[21]  Nagarajan Natarajan,et al.  Inductive matrix completion for predicting gene–disease associations , 2014, Bioinform..

[22]  Alexandre Bernardino,et al.  Matrix Completion for Multi-label Image Classification , 2011, NIPS.

[23]  Razvan Sultana,et al.  Genomic Analysis Identifies Targets of Convergent Positive Selection in Drug Resistant Mycobacterium tuberculosis , 2013, Nature Genetics.

[24]  D. Gabay Applications of the method of multipliers to variational inequalities , 1983 .

[25]  Quentin Berthet,et al.  Optimal link prediction with matrix logistic regression , 2018, 1803.07054.

[26]  Inderjit S. Dhillon,et al.  Large-scale Multi-label Learning with Missing Labels , 2013, ICML.

[27]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[28]  Georgios B. Giannakis,et al.  Consensus-Based Distributed Support Vector Machines , 2010, J. Mach. Learn. Res..

[29]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[30]  Mohamed-Jalal Fadili,et al.  Monotone operator splitting for optimization problems in sparse recovery , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[31]  Noah Simon,et al.  A Sparse-Group Lasso , 2013 .

[32]  Phelim Bradley,et al.  Whole-genome sequencing for prediction of Mycobacterium tuberculosis drug susceptibility and resistance: a retrospective cohort study , 2015, The Lancet. Infectious diseases.

[33]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[34]  Inderjit S. Dhillon,et al.  Matrix Completion with Noisy Side Information , 2015, NIPS.

[35]  Inderjit S. Dhillon,et al.  Provable Inductive Matrix Completion , 2013, ArXiv.

[36]  Xin Wang,et al.  Low-rank matrix completion for array signal processing , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[37]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[38]  Jinfeng Yi,et al.  Robust Ensemble Clustering by Matrix Completion , 2012, 2012 IEEE 12th International Conference on Data Mining.

[39]  Ohad Shamir,et al.  Matrix completion with the trace norm: learning, bounding, and transducing , 2014, J. Mach. Learn. Res..

[40]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[41]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[42]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..