Generalized Embedding Machines for Recommender Systems

Factorization machine (FM) is an effective model for feature-based recommendation which utilizes inner product to capture second-order feature interactions. However, one of the major drawbacks of FM is that it couldn't capture complex high-order interaction signals. A common solution is to change the interaction function, such as stacking deep neural networks on the top of FM. In this work, we propose an alternative approach to model high-order interaction signals in the embedding level, namely Generalized Embedding Machine (GEM). The embedding used in GEM encodes not only the information from the feature itself but also the information from other correlated features. Under such situation, the embedding becomes high-order. Then we can incorporate GEM with FM and even its advanced variants to perform feature interactions. More specifically, in this paper we utilize graph convolution networks (GCN) to generate high-order embeddings. We integrate GEM with several FM-based models and conduct extensive experiments on two real-world datasets. The results demonstrate significant improvement of GEM over corresponding baselines.

[1]  Yantao Yu,et al.  An Input-aware Factorization Machine for Sparse Prediction , 2019, IJCAI.

[2]  Steffen Rendle,et al.  Factorization Machines , 2010, 2010 IEEE International Conference on Data Mining.

[3]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[4]  Tat-Seng Chua,et al.  Neural Factorization Machines for Sparse Predictive Analytics , 2017, SIGIR.

[5]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[6]  J. Weston,et al.  Support vector regression with ANOVA decomposition kernels , 1999 .

[7]  F. Maxwell Harper,et al.  The MovieLens Datasets: History and Context , 2016, TIIS.

[8]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[9]  Heng-Tze Cheng,et al.  Wide & Deep Learning for Recommender Systems , 2016, DLRS@RecSys.

[10]  Ge Chen,et al.  Interaction-aware Factorization Machines for Recommender Systems , 2019, AAAI.

[11]  Gang Fu,et al.  Deep & Cross Network for Ad Click Predictions , 2017, ADKDD@KDD.

[12]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[13]  Tat-Seng Chua,et al.  Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks , 2017, IJCAI.

[14]  Bin Yu,et al.  Boosting with early stopping: Convergence and consistency , 2005, math/0508276.

[15]  Ruocheng Guo,et al.  Strongly Hierarchical Factorization Machines and ANOVA Kernel Regression , 2017, SDM.

[16]  Philip S. Yu,et al.  Multi-view Machines , 2015, WSDM.

[17]  Steffen Rendle,et al.  Factorization Machines with libFM , 2012, TIST.

[18]  Li Shen,et al.  A Sufficient Condition for Convergences of Adam and RMSProp , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Xiangnan He,et al.  CFM: Convolutional Factorization Machines for Context-Aware Recommendation , 2019, IJCAI.

[20]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[21]  Jason Weston,et al.  Memory Networks , 2014, ICLR.

[22]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[23]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  AdomaviciusGediminas,et al.  Toward the Next Generation of Recommender Systems , 2005 .

[25]  Yunming Ye,et al.  DeepFM: A Factorization-Machine based Neural Network for CTR Prediction , 2017, IJCAI.

[26]  A KonstanJoseph,et al.  The MovieLens Datasets , 2015 .

[27]  Philip S. Yu,et al.  Multilinear Factorization Machines for Multi-Task Multi-View Learning , 2017, WSDM.

[28]  Naonori Ueda,et al.  Polynomial Networks and Factorization Machines: New Insights and Efficient Training Algorithms , 2016, ICML.

[29]  Naonori Ueda,et al.  Higher-Order Factorization Machines , 2016, NIPS.

[30]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[31]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[32]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[33]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[34]  Nuria Oliver,et al.  Frappe: Understanding the Usage and Perception of Mobile App Recommendations In-The-Wild , 2015, ArXiv.