Variational Session-based Recommendation Using Normalizing Flows

We present a novel generative Session-Based Recommendation (SBR) framework, called VAriational SEssion-based Recommendation (VASER) - a non-linear probabilistic methodology allowing Bayesian inference for flexible parameter estimation of sequential recommendations. Instead of directly applying extended Variational AutoEncoders (VAE) to SBR, the proposed method introduces normalizing flows to estimate the probabilistic posterior, which is more effective than the agnostic presumed prior approximation used in existing deep generative recommendation approaches. VASER explores soft attention mechanism to upweight the important clicks in a session. We empirically demonstrate that the proposed model significantly outperforms several state-of-the-art baselines, including the recently-proposed RNN/VAE-based approaches on real-world datasets.

[1]  Shakir Mohamed,et al.  Variational Inference with Normalizing Flows , 2015, ICML.

[2]  George Karypis,et al.  Item-based top-N recommendation algorithms , 2004, TOIS.

[3]  David Duvenaud,et al.  Inference Suboptimality in Variational Autoencoders , 2018, ICML.

[4]  Sheng Li,et al.  Deep Collaborative Filtering via Marginalized Denoising Auto-encoder , 2015, CIKM.

[5]  Dietmar Jannach,et al.  When Recurrent Neural Networks meet the Neighborhood for Session-Based Recommendation , 2017, RecSys.

[6]  Zhaochun Ren,et al.  Neural Attentive Session-based Recommendation , 2017, CIKM.

[7]  Alexandros Karatzoglou,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.

[8]  Dit-Yan Yeung,et al.  Collaborative Deep Learning for Recommender Systems , 2014, KDD.

[9]  Sotirios Chatzis,et al.  Recurrent Latent Variable Networks for Session-Based Recommendation , 2017, DLRS@RecSys.

[10]  Samy Bengio,et al.  Generating Sentences from a Continuous Space , 2015, CoNLL.

[11]  Enhong Chen,et al.  Learning from History and Present: Next-item Recommendation via Discriminatively Exploiting User Behaviors , 2018, KDD.

[12]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[13]  Yoshua Bengio,et al.  Z-Forcing: Training Stochastic Recurrent Networks , 2017, NIPS.

[14]  Geoffrey E. Hinton,et al.  Restricted Boltzmann machines for collaborative filtering , 2007, ICML '07.

[15]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[16]  Chen Liu,et al.  Modeling User Session and Intent with an Attention-based Encoder-Decoder Architecture , 2017, RecSys.

[17]  M. de Rijke,et al.  A Collective Variational Autoencoder for Top-N Recommendation with Side Information , 2018, DLRS@RecSys.

[18]  Lars Schmidt-Thieme,et al.  Factorizing personalized Markov chains for next-basket recommendation , 2010, WWW '10.

[19]  Boi Faltings,et al.  Context Tree for Adaptive Session-based Recommendation , 2018, ArXiv.

[20]  Julian J. McAuley,et al.  Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[21]  Yong Yu,et al.  SVDFeature: a toolkit for feature-based collaborative filtering , 2012, J. Mach. Learn. Res..

[22]  Eric P. Xing,et al.  Toward Controlled Generation of Text , 2017, ICML.

[23]  Alexandros Karatzoglou,et al.  Recurrent Neural Networks with Top-k Gains for Session-based Recommendations , 2017, CIKM.

[24]  Ke Wang,et al.  Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding , 2018, WSDM.

[25]  U. V. Luxburg,et al.  Improving Variational Autoencoders with Inverse Autoregressive Flow , 2016 .

[26]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[27]  Pascal Poupart,et al.  Variational Attention for Sequence-to-Sequence Models , 2017, COLING.

[28]  Qiang Gao,et al.  Identifying Human Mobility via Trajectory Embeddings , 2017, IJCAI.

[29]  Dit-Yan Yeung,et al.  Collaborative Recurrent Autoencoder: Recommend while Learning to Fill in the Blanks , 2016, NIPS.

[30]  Daan Wierstra,et al.  Stochastic Backpropagation and Approximate Inference in Deep Generative Models , 2014, ICML.

[31]  Alexandros Karatzoglou,et al.  Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks , 2017, RecSys.

[32]  Alexandros Karatzoglou,et al.  Parallel Recurrent Neural Network Architectures for Feature-rich Session-based Recommendations , 2016, RecSys.

[33]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[34]  Qiao Liu,et al.  STAMP: Short-Term Attention/Memory Priority Model for Session-based Recommendation , 2018, KDD.

[35]  Adilson Marques da Cunha,et al.  News Session-Based Recommendations using Deep Neural Networks , 2018, DLRS@RecSys.

[36]  Joemon M. Jose,et al.  A Simple but Hard-to-Beat Baseline for Session-based Recommendations , 2018, ArXiv.

[37]  Martin Ester,et al.  Collaborative Denoising Auto-Encoders for Top-N Recommender Systems , 2016, WSDM.

[38]  Qiang Gao,et al.  Trajectory-User Linking via Variational AutoEncoder , 2018, IJCAI.

[39]  Kyungwoo Song,et al.  Augmented Variational Autoencoders for Collaborative Filtering with Auxiliary Information , 2017, CIKM.

[40]  Weinan Zhang,et al.  LambdaFM: Learning Optimal Ranking with Factorization Machines Using Lambda Surrogates , 2016, CIKM.

[41]  Yong Liu,et al.  Improved Recurrent Neural Networks for Session-based Recommendations , 2016, DLRS@RecSys.

[42]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[43]  Matthew D. Hoffman,et al.  On the challenges of learning with inference networks on sparse, high-dimensional data , 2017, AISTATS.

[44]  Pengfei Wang,et al.  Learning Hierarchical Representation Model for NextBasket Recommendation , 2015, SIGIR.

[45]  Alexander M. Rush,et al.  Latent Alignment and Variational Attention , 2018, NeurIPS.

[46]  Scott Sanner,et al.  AutoRec: Autoencoders Meet Collaborative Filtering , 2015, WWW.

[47]  Feng Liu,et al.  An Adjustable Heat Conduction based KNN Approach for Session-based Recommendation , 2018, ArXiv.

[48]  Dietmar Jannach,et al.  Sequence-Aware Recommender Systems , 2018, UMAP.

[49]  Ke Zhang,et al.  Variational Recurrent Model for Session-based Recommendation , 2018, CIKM.

[50]  James She,et al.  Collaborative Variational Autoencoder for Recommender Systems , 2017, KDD.

[51]  Jie Yuan,et al.  Item Recommendation with Variational Autoencoders and Heterogenous Priors , 2018, ArXiv.

[52]  Max Welling,et al.  Improved Variational Inference with Inverse Autoregressive Flow , 2016, NIPS 2016.

[53]  Pieter Abbeel,et al.  Variational Lossy Autoencoder , 2016, ICLR.

[54]  Jie Yuan,et al.  Item Recommendation with Variational Autoencoders and Heterogeneous Priors , 2018, DLRS@RecSys.

[55]  Matthew D. Hoffman,et al.  Variational Autoencoders for Collaborative Filtering , 2018, WWW.