Masked and Swapped Sequence Modeling for Next Novel Basket Recommendation in Grocery Shopping

Next basket recommendation (NBR) is the task of predicting the next set of items based on a sequence of already purchased baskets. It is a recommendation task that has been widely studied, especially in the context of grocery shopping. In next basket recommendation (NBR), it is useful to distinguish between repeat items, i.e., items that a user has consumed before, and explore items, i.e., items that a user has not consumed before. Most NBR work either ignores this distinction or focuses on repeat items. We formulate the next novel basket recommendation (NNBR) task, i.e., the task of recommending a basket that only consists of novel items, which is valuable for both real-world application and NBR evaluation. We evaluate how existing NBR methods perform on the NNBR task and find that, so far, limited progress has been made w.r.t. the NNBR task. To address the NNBR task, we propose a simple bi-directional transformer basket recommendation model (BTBR), which is focused on directly modeling item-to-item correlations within and across baskets instead of learning complex basket representations. To properly train BTBR, we propose and investigate several masking strategies and training objectives: (i) item-level random masking, (ii) item-level select masking, (iii) basket-level all masking, (iv) basket-level explore masking, and (v) joint masking. In addition, an item-basket swapping strategy is proposed to enrich the item interactions within the same baskets. We conduct extensive experiments on three open datasets with various characteristics. The results demonstrate the effectiveness of BTBR and our masking and swapping strategies for the NNBR task. BTBR with a properly selected masking and swapping strategy can substantially improve NNBR performance.

[1]  Oren Barkan,et al.  Learning to Ride a Buy-Cycle: A Hyper-Convolutional Model for Next Basket Repurchase Recommendation , 2022, RecSys.

[2]  M. de Rijke,et al.  ReCANet: A Repeat Consumption-Aware Neural Network for Next Basket Recommendation in Grocery Shopping , 2022, SIGIR.

[3]  Lingyang Chu,et al.  Flexible Order Aware Sequential Recommendation , 2022, ICMR.

[4]  Jinyang Gao,et al.  Contrastive Learning for Sequential Recommendation , 2022, 2022 IEEE 38th International Conference on Data Engineering (ICDE).

[5]  Hongzhi Yin,et al.  Self-Supervised Learning for Recommender Systems: A Survey , 2022, IEEE Transactions on Knowledge and Data Engineering.

[6]  Enhong Chen,et al.  Learning Transferable User Representations with Sequential Behaviors via Contrastive Pre-training , 2021, 2021 IEEE International Conference on Data Mining (ICDM).

[7]  M. de Rijke,et al.  A Next Basket Recommendation Reality Check , 2021, ACM Trans. Inf. Syst..

[8]  Julian McAuley,et al.  Modeling Dynamic Attributes for Next Basket Recommendation , 2021, ArXiv.

[9]  Vojtech Vancura,et al.  Neural Basket Embedding for Sequential Recommendation , 2021, RecSys.

[10]  Chenliang Li,et al.  The World is Binary: Contrastive Learning for Denoising Next Basket Recommendation , 2021, SIGIR.

[11]  Li Yu,et al.  Recurrent Convolution Basket Map for Diversity Next-Basket Recommendation , 2020, DASFAA.

[12]  Weifeng Lv,et al.  Dual Sequential Network for Temporal Sets Prediction , 2020, SIGIR.

[13]  Mirko Polato,et al.  Recency Aware Collaborative Filtering for Next Basket Recommendation , 2020, UMAP.

[14]  Hui Xiong,et al.  Predicting Temporal Sets with Deep Neural Networks , 2020, KDD.

[15]  Xiangnan He,et al.  Modeling Personalized Item Frequency Information for Next-basket Recommendation , 2020, SIGIR.

[16]  Quan Z. Sheng,et al.  Intention Nets: Psychology-Inspired User Choice Behavior Modeling for Next-Basket Prediction , 2020, AAAI.

[17]  Pengfei Wang,et al.  Modeling Temporal Dynamics of Users’ Purchase Behaviors for Next Basket Prediction , 2019, Journal of Computer Science and Technology.

[18]  Duc-Trong Le,et al.  Correlation-Sensitive Next-Basket Recommendation , 2019, IJCAI.

[19]  Quan Z. Sheng,et al.  Sequential Recommender Systems: Challenges, Progress and Prospects , 2019, IJCAI.

[20]  Xiangnan He,et al.  Sets2Sets: Learning from Sequential Sets with Neural Networks , 2019, KDD.

[21]  Dietmar Jannach,et al.  Are we really making much progress? A worrying analysis of recent neural recommendation approaches , 2019, RecSys.

[22]  Peng Jiang,et al.  BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer , 2019, CIKM.

[23]  Xing Xie,et al.  Session-based Recommendation with Graph Neural Networks , 2018, AAAI.

[24]  Jie Liu,et al.  Representing and Recommending Shopping Baskets with Complementarity, Compatibility and Loyalty , 2018, CIKM.

[25]  Julian J. McAuley,et al.  Self-Attentive Sequential Recommendation , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[26]  Qiao Liu,et al.  STAMP: Short-Term Attention/Memory Priority Model for Session-based Recommendation , 2018, KDD.

[27]  Ji-Rong Wen,et al.  An Attribute-aware Neural Attentive Model for Next Basket Recommendation , 2018, SIGIR.

[28]  Ke Wang,et al.  Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding , 2018, WSDM.

[29]  Zhaochun Ren,et al.  Neural Attentive Session-based Recommendation , 2017, CIKM.

[30]  Alexandros Karatzoglou,et al.  Recurrent Neural Networks with Top-k Gains for Session-based Recommendations , 2017, CIKM.

[31]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[32]  Derek Bridge,et al.  Diversity, Serendipity, Novelty, and Coverage , 2016, ACM Trans. Interact. Intell. Syst..

[33]  Geoffrey E. Hinton,et al.  Layer Normalization , 2016, ArXiv.

[34]  Feng Yu,et al.  A Dynamic Recurrent Model for Next Basket Recommendation , 2016, SIGIR.

[35]  Kevin Gimpel,et al.  Bridging Nonlinearities and Stochastic Regularizers with Gaussian Error Linear Units , 2016, ArXiv.

[36]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Balázs Hidasi,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.

[38]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[39]  Lars Schmidt-Thieme,et al.  Factorizing personalized Markov chains for next-basket recommendation , 2010, WWW '10.

[40]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.

[41]  Wilson L. Taylor,et al.  “Cloze Procedure”: A New Tool for Measuring Readability , 1953 .

[42]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[43]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[44]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[45]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.