Multi-Interest Multi-Round Conversational Recommendation System with Fuzzy Feedback based User Simulator

Conversational recommendation system (CRS) is able to obtain fine-grained and dynamic user preferences based on interactive dialogue. Previous CRS assumes that the user has a clear target item, which often deviates from the real scenario. The user may have a clear single preference for some attribute types (e.g. brand) of items, while for other attribute types (e.g. color), the user may have multiple preferences or even no clear preferences, which leads to multiple acceptable items under multiple combinations of attribute instances. Furthermore, previous works assume that users would provide clear responses to any questions asked by the system. And they also assume that users would be dedicated to the target item, that is, user would answer ”yes” to the attribute corresponding to the target item and answer ”no” to other attributes. However, users’ responses to attributes are not completely dependent on target items, but also influenced by users’ inherent interests. Besides, for some over-specific or equivocal questions, the feedback of user might not be clear (”yes”/”no”) and user might give some fuzzy response like ”I don’t know”. To address the aforementioned issues, we first propose a more realistic conversational recommendation learning setting, namely Multi-Interest Multi-round Conversational Recommendation (MIMCR), where users may have multiple interests in attribute instance combinations and accept multiple items with partially overlapped combinations of attribute instances. To effectively cope with MIMCR, we propose a novel learning framework, namely Multiple Choice questions based Multi-Interest Policy Learning. Moreover, we further propose a more realistic User-centric User Simulator with Fuzzy Feedback (UUSFF), which naturally calibrates the user response with additional fuzzy feedback based on user‘s inherent preference. To better match the new scenario UUSFF, we propose a simple but effective adaption method for different backbones. Extensive experimental results on several datasets demonstrate the superiority of our methods for the proposed settings.

[1]  Lingfei Wu,et al.  Graph Neural Networks: Foundation, Frontiers and Applications , 2023, KDD.

[2]  Liang Zhao,et al.  Graph Neural Networks: Foundation, Frontiers and Applications , 2022, KDD.

[3]  Julian McAuley,et al.  Bundle MCR: Towards Conversational Bundle Recommendation , 2022, RecSys.

[4]  Yubao Liu,et al.  Learning to Infer User Implicit Preference in Conversational Recommendation , 2022, SIGIR.

[5]  Xiting Wang,et al.  Multi-level Recommendation Reasoning over Knowledge Graphs with Reinforcement Learning , 2022, WWW.

[6]  Zhihua Wei,et al.  Multiple Choice Questions based Multi-Interest Policy Learning for Conversational Recommendation , 2021, WWW.

[7]  Fangli Xu,et al.  Graph Learning Augmented Heterogeneous Graph Neural Network for Social Recommendation , 2021, Trans. Recomm. Syst..

[8]  Fangli Xu,et al.  Multi-behavior Graph Contextual Aware Network for Session-based Recommendation , 2021, ArXiv.

[9]  Shuai Li,et al.  Comparison-based Conversational Recommender System with Relative Bandit Feedback , 2021, SIGIR.

[10]  Chao Huang,et al.  Graph Meta Network for Multi-Behavior Recommendation , 2021, SIGIR.

[11]  Zhihua Wei,et al.  Heterogeneous Global Graph Neural Networks for Personalized Session-based Recommendation , 2021, WSDM.

[12]  Kai Zheng,et al.  Learning to Ask Appropriate Questions in Conversational Recommendation , 2021, SIGIR.

[13]  Danqi Chen,et al.  SimCSE: Simple Contrastive Learning of Sentence Embeddings , 2021, EMNLP.

[14]  Ji-rong Wen,et al.  Adapting User Preference to Online Feedback in Multi-round Conversational Recommendation , 2021, WSDM.

[15]  Ming Zhang,et al.  DisenHAN: Disentangled Heterogeneous Graph Attention Network for Recommendation , 2020, CIKM.

[16]  Wayne Xin Zhao,et al.  Towards Topic-Guided Conversational Recommender System , 2020, COLING.

[17]  Xiangnan He,et al.  MGAT: Multimodal Graph Attention Network for Recommendation , 2020, Inf. Process. Manag..

[18]  M. de Rijke,et al.  Conversational Recommendation: Formulation, Methods, and Evaluation , 2020, SIGIR.

[19]  Yu Fan,et al.  KERL: A Knowledge-Guided Reinforcement Learning Model for Sequential Recommendation , 2020, SIGIR.

[20]  Qing Li,et al.  A Graph Neural Network Framework for Social Recommendations , 2020, IEEE Transactions on Knowledge and Data Engineering.

[21]  Kun Zhou,et al.  Improving Conversational Recommender Systems via Knowledge Graph based Semantic Fusion , 2020, KDD.

[22]  Yulong Gu,et al.  Neural Interactive Collaborative Filtering , 2020, SIGIR.

[23]  Xiangnan He,et al.  Disentangled Graph Collaborative Filtering , 2020, SIGIR.

[24]  Xiangnan He,et al.  Interactive Path Reasoning on Graph for Conversational Recommendation , 2020, KDD.

[25]  Weinan Zhang,et al.  Interactive Recommender System via Knowledge Graph-enhanced Reinforcement Learning , 2020, SIGIR.

[26]  Mohammed J. Zaki,et al.  Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings , 2020, NeurIPS.

[27]  Evangelos Kanoulas,et al.  Towards Question-based Recommender Systems , 2020, SIGIR.

[28]  Xiangnan He,et al.  Seamlessly Unifying Attributes and Items: Conversational Recommendation for Cold-start Users , 2020, ACM Trans. Inf. Syst..

[29]  Chang Zhou,et al.  Controllable Multi-Interest Framework for Recommendation , 2020, KDD.

[30]  Zheng-Yu Niu,et al.  Towards Conversational Recommendation over Multi-Type Dialogs , 2020, ACL.

[31]  Xiangnan He,et al.  LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation , 2020, SIGIR.

[32]  Xiangnan He,et al.  Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems , 2020, WSDM.

[33]  Dawei Yin,et al.  Pseudo Dyna-Q: A Reinforcement Learning Framework for Interactive Recommendation , 2020, WSDM.

[34]  Mohammed J. Zaki,et al.  Reinforcement Learning Based Graph-to-Sequence Model for Natural Question Generation , 2019, ICLR.

[35]  Hongxia Yang,et al.  Towards Knowledge-Based Recommender Dialog System , 2019, EMNLP.

[36]  Yongliang Li,et al.  Metapath-guided Heterogeneous Graph Neural Network for Intent Recommendation , 2019, KDD.

[37]  Hyunsouk Cho,et al.  MeLU: Meta-Learned User Preference Estimator for Cold-Start Recommendation , 2019, KDD.

[38]  Zhou Yu,et al.  Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good , 2019, ACL.

[39]  Xiyuan Zhang,et al.  Proactive Human-Machine Conversation with Explicit Conversation Goal , 2019, ACL.

[40]  Tat-Seng Chua,et al.  Neural Graph Collaborative Filtering , 2019, SIGIR.

[41]  Yixin Cao,et al.  KGAT: Knowledge Graph Attention Network for Recommendation , 2019, KDD.

[42]  Xiao Lin,et al.  Value-aware Recommendation based on Reinforcement Profit Maximization , 2019, WWW.

[43]  Rui Yan,et al.  Deep Chit-Chat: Deep Learning for Chatbots , 2019, WWW.

[44]  Guihai Chen,et al.  Dual Graph Attention Networks for Deep Latent Representation of Multifaceted Social Effects in Recommender Systems , 2019, WWW.

[45]  Minyi Guo,et al.  Knowledge Graph Convolutional Networks for Recommender Systems , 2019, WWW.

[46]  Yuan He,et al.  Graph Neural Networks for Social Recommendation , 2019, WWW.

[47]  Yixin Cao,et al.  Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences , 2019, WWW.

[48]  Zhiyuan Liu,et al.  OpenKE: An Open Toolkit for Knowledge Embedding , 2018, EMNLP.

[49]  Christopher Joseph Pal,et al.  Towards Deep Conversational Recommendations , 2018, NeurIPS.

[50]  Xu Chen,et al.  Towards Conversational Search and Recommendation: System Ask, User Respond , 2018, CIKM.

[51]  Filip Radlinski,et al.  Preference elicitation as an optimization problem , 2018, RecSys.

[52]  Min-Yen Kan,et al.  Sequicity: Simplifying Task-oriented Dialogue Systems with Single Sequence-to-Sequence Architectures , 2018, ACL.

[53]  Yi Zhang,et al.  Conversational Recommender System , 2018, SIGIR.

[54]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[55]  Nicholas Jing Yuan,et al.  DRN: A Deep Reinforcement Learning Framework for News Recommendation , 2018, WWW.

[56]  Minyi Guo,et al.  RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems , 2018, CIKM.

[57]  Liang Zhang,et al.  Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning , 2018, KDD.

[58]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[59]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[60]  Yann Dauphin,et al.  Deal or No Deal? End-to-End Learning of Negotiation Dialogues , 2017, EMNLP.

[61]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[62]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[63]  Filip Radlinski,et al.  Towards Conversational Recommender Systems , 2016, KDD.

[64]  Tom Schaul,et al.  Dueling Network Architectures for Deep Reinforcement Learning , 2015, ICML.

[65]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[66]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[67]  Yiqun Liu,et al.  Do users rate or review?: boost phrase-level sentiment labeling with review-level sentiment classification , 2014, SIGIR.

[68]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[69]  Guy Shani,et al.  An MDP-Based Recommender System , 2002, J. Mach. Learn. Res..

[70]  Robert E. Kalaba,et al.  On the role of dynamic programming in statistical communication theory , 1957, IRE Trans. Inf. Theory.

[71]  Kam-Fai Wong,et al.  Finetuning Large-Scale Pre-trained Language Models for Conversational Recommendation with Knowledge Graph , 2021, ArXiv.

[72]  C. Vong,et al.  Improving Conversational Recommendation System by Pretraining on Billions Scale of Knowledge Graph , 2021, ArXiv.

[73]  Hongxia Jin,et al.  Text-Based Interactive Recommendation via Constraint-Augmented Reinforcement Learning , 2019, NeurIPS.

[74]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.