Advances and Challenges in Conversational Recommender Systems: A Survey

Recommender systems exploit interaction history to estimate user preference, having been heavily used in a wide range of industry applications. However, static recommendation models are difficult to answer two important questions well due to inherent shortcomings: (a) What exactly does a user like? (b) Why does a user like an item? The shortcomings are due to the way that static models learn user preference, i.e., without explicit instructions and active feedback from users. The recent rise of conversational recommender systems (CRSs) changes this situation fundamentally. In a CRS, users and the system can dynamically communicate through natural language interactions, which provide unprecedented opportunities to explicitly obtain the exact preference of users. Considerable efforts, spread across disparate settings and applications, have been put into developing CRSs. Existing models, technologies, and evaluation methods for CRSs are far from mature. In this paper, we provide a systematic review of the techniques used in current CRSs. We summarize the key challenges of developing CRSs into five directions: (1) Question-based user preference elicitation. (2) Multi-turn conversational recommendation strategies. (3) Dialogue understanding and generation. (4) Exploitation-exploration trade-offs. (5) Evaluation and user simulation. These research directions involve multiple research fields like information retrieval (IR), natural language processing (NLP), and human-computer interaction (HCI). Based on these research directions, we discuss some future challenges and opportunities. We provide a road map for researchers from multiple communities to get started in this area. We hope this survey helps to identify and address challenges in CRSs and inspire future research.

[1]  M. de Rijke,et al.  Wizard of Search Engine: Access to Information Through Conversations with Search Engines , 2021, SIGIR.

[2]  Zheng Wen,et al.  Cascading Bandits: Learning to Rank in the Cascade Model , 2015, ICML.

[3]  Mingbo Ma,et al.  When to Finish? Optimal Beam Search for Neural Text Generation (modulo beam size) , 2017, EMNLP.

[4]  Yongfeng Zhang,et al.  COOKIE: A Dataset for Conversational Recommendation over Knowledge Graphs in E-commerce , 2020, ArXiv.

[5]  Rui Yan,et al.  Deep Chit-Chat: Deep Learning for Chatbots , 2019, WWW.

[6]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[7]  Yulong Gu,et al.  Neural Interactive Collaborative Filtering , 2020, SIGIR.

[8]  Elizabeth Clark,et al.  Evaluation of Text Generation: A Survey , 2020, ArXiv.

[9]  Hongxia Jin,et al.  A Visual Dialog Augmented Interactive Recommender System , 2019, KDD.

[10]  Xiaoying Zhang,et al.  Conversational Contextual Bandit: Algorithm and Application , 2020, WWW.

[11]  Michael Arens,et al.  Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Survey , 2019, Mach. Learn. Knowl. Extr..

[12]  Enhong Chen,et al.  Personalized Ranking with Importance Sampling , 2020, WWW.

[13]  Yongdong Zhang,et al.  LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation , 2020, SIGIR.

[14]  Geoffrey Zweig,et al.  Recurrent neural networks for language understanding , 2013, INTERSPEECH.

[15]  Olfa Nasraoui,et al.  Debiasing the Human-Recommender System Feedback Loop in Collaborative Filtering , 2019, WWW.

[16]  Maarten de Rijke,et al.  Keeping Dataset Biases out of the Simulation , 2020 .

[17]  Xing Xie,et al.  A Survey on Knowledge Graph-Based Recommender Systems , 2020, IEEE Transactions on Knowledge and Data Engineering.

[18]  Xu Chen,et al.  Explainable Recommendation: A Survey and New Perspectives , 2018, Found. Trends Inf. Retr..

[19]  Mirella Lapata,et al.  Probabilistic Text Structuring: Experiments with Sentence Ordering , 2003, ACL.

[20]  Xiang Zhang,et al.  Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systems , 2015, ICLR.

[21]  Enrique Alfonseca,et al.  Learning to Attend, Copy, and Generate for Session-Based Query Suggestion , 2017, CIKM.

[22]  Lihong Li,et al.  An Empirical Evaluation of Thompson Sampling , 2011, NIPS.

[23]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[24]  Feng Huang,et al.  Deep reinforcement learning: a survey , 2020, Frontiers of Information Technology & Electronic Engineering.

[25]  Jun Wang,et al.  Interactive collaborative filtering , 2013, CIKM.

[26]  Ziv Bar-Yossef,et al.  Context-sensitive query auto-completion , 2011, WWW.

[27]  Anmol Bhasin,et al.  From Infrastructure to Culture: A/B Testing Challenges in Large Scale Social Networks , 2015, KDD.

[28]  Philip S. Yu,et al.  Leveraging Meta-path based Context for Top- N Recommendation with A Neural Co-Attention Model , 2018, KDD.

[29]  Scott Sanner,et al.  Real-time Multiattribute Bayesian Preference Elicitation with Pairwise Comparison Queries , 2010, AISTATS.

[30]  Susan T. Dumais,et al.  Short-Term Satisfaction and Long-Term Coverage: Understanding How Users Tolerate Algorithmic Exploration , 2018, WSDM.

[31]  Zheng Wen,et al.  Cascading Bandits for Large-Scale Recommendation Problems , 2016, UAI.

[32]  Krisztian Balog,et al.  IAI MovieBot: A Conversational Movie Recommender System , 2020, CIKM.

[33]  Min Gao,et al.  Generating Reliable Friends via Adversarial Training to Improve Social Recommendation , 2019, 2019 IEEE International Conference on Data Mining (ICDM).

[34]  Krisztian Balog,et al.  Evaluating Conversational Recommender Systems via User Simulation , 2020, KDD.

[35]  Yongfeng Zhang,et al.  Reinforcement Knowledge Graph Reasoning for Explainable Recommendation , 2019, SIGIR.

[36]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[37]  Xiangnan He,et al.  Bias and Debias in Recommender System: A Survey and Future Directions , 2020, ACM Trans. Inf. Syst..

[38]  Jiliang Tang,et al.  Toward Simulating Environments in Reinforcement Learning Based Recommendations , 2019, ArXiv.

[39]  Jiliang Tang,et al.  A Survey on Dialogue Systems: Recent Advances and New Frontiers , 2017, SKDD.

[40]  Shiwen Wu,et al.  Graph Neural Networks in Recommender Systems: A Survey , 2020, ArXiv.

[41]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[42]  Zhijian Ou,et al.  Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context , 2019, AAAI.

[43]  Chunyan Miao,et al.  A Hybrid Bandit Framework for Diversified Recommendation , 2020, AAAI.

[44]  Barry Smyth,et al.  Thinking Positively - Explanatory Feedback for Conversational Recommender Systems , 2004 .

[45]  Wei Chu,et al.  A contextual-bandit approach to personalized news article recommendation , 2010, WWW '10.

[46]  Minlie Huang,et al.  Bridging the Gap between Conversational Reasoning and Interactive Recommendation , 2020, ArXiv.

[47]  Hongxia Jin,et al.  Reward Constrained Interactive Recommendation with Natural Language Feedback , 2020, ArXiv.

[48]  Xiaoyan Zhu,et al.  Commonsense Knowledge Aware Conversation Generation with Graph Attention , 2018, IJCAI.

[49]  Pararth Shah,et al.  Recommendation as a Communication Game: Self-Supervised Bot-Play for Goal-oriented Dialogue , 2019, EMNLP.

[50]  Zheng-Yu Niu,et al.  Towards Conversational Recommendation over Multi-Type Dialogs , 2020, ACL.

[51]  Jianfeng Gao,et al.  Neural Approaches to Conversational AI: Question Answering, Task-oriented Dialogues and Social Chatbots , 2019 .

[52]  Yujing Hu,et al.  Reinforcement Learning to Rank in E-Commerce Search Engine: Formalization, Analysis, and Application , 2018, KDD.

[53]  Ben Carterette,et al.  Counterfactual Evaluation of Slate Recommendations with Sequential Reward Interactions , 2020, KDD.

[54]  Arpit Rana,et al.  Navigation-by-preference: a new conversational recommender with preference-based feedback , 2020, IUI.

[55]  Jun Zhao,et al.  Inner Attention based Recurrent Neural Networks for Answer Selection , 2016, ACL.

[56]  Justine Cassell,et al.  A Model of Social Explanations for a Conversational Movie Recommendation System , 2019, HAI.

[57]  Hongning Wang,et al.  Model-Based Reinforcement Learning with Adversarial Training for Online Recommendation , 2019, NeurIPS.

[58]  Chang Zhou,et al.  Controllable Multi-Interest Framework for Recommendation , 2020, KDD.

[59]  Hongxia Yang,et al.  Learning Disentangled Representations for Recommendation , 2019, NeurIPS.

[60]  Fuzheng Zhang,et al.  Leveraging Historical Interaction Data for Improving Conversational Recommender System , 2020, CIKM.

[61]  Xiaoyan Zhu,et al.  Generating Informative Responses with Controlled Sentence Function , 2018, ACL.

[62]  Yiqun Liu,et al.  How good your recommender system is? A survey on evaluations in recommendation , 2017, International Journal of Machine Learning and Cybernetics.

[63]  Geoffrey Zweig,et al.  Spoken language understanding using long short-term memory neural networks , 2014, 2014 IEEE Spoken Language Technology Workshop (SLT).

[64]  Verena Rieser,et al.  Why We Need New Evaluation Metrics for NLG , 2017, EMNLP.

[65]  Erik Cambria,et al.  Augmenting End-to-End Dialogue Systems With Commonsense Knowledge , 2018, AAAI.

[66]  Li Chen,et al.  Critiquing-based recommenders: survey and emerging trends , 2012, User Modeling and User-Adapted Interaction.

[67]  M. de Rijke,et al.  An Analysis of Mixed Initiative and Collaboration in Information-Seeking Dialogues , 2020, SIGIR.

[68]  Xiyuan Zhang,et al.  Proactive Human-Machine Conversation with Explicit Conversation Goal , 2019, ACL.

[69]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[70]  C. Hauff,et al.  What does BERT know about books, movies and music? Probing BERT for Conversational Recommendation , 2020, RecSys.

[71]  Huazheng Wang,et al.  Factorization Bandits for Interactive Recommendation , 2017, AAAI.

[72]  D. Hensher,et al.  Stated Choice Methods: Analysis and Applications , 2000 .

[73]  Yong Yu,et al.  Are You Influenced by Others When Rating?: Improve Rating Prediction by Conformity Modeling , 2016, RecSys.

[74]  Yu Zhang,et al.  Personalizing a Dialogue System With Transfer Reinforcement Learning , 2016, AAAI.

[75]  Xiangnan He,et al.  Disentangled Graph Collaborative Filtering , 2020, SIGIR.

[76]  Boi Faltings,et al.  Conversational recommenders with adaptive suggestions , 2007, RecSys '07.

[77]  Zhou Yu,et al.  Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good , 2019, ACL.

[78]  Dawei Yin,et al.  Pseudo Dyna-Q: A Reinforcement Learning Framework for Interactive Recommendation , 2020, WSDM.

[79]  Ji-Rong Wen,et al.  Adapting User Preference to Online Feedback in Multi-round Conversational Recommendation , 2021, WSDM.

[80]  Nicholas Jing Yuan,et al.  DRN: A Deep Reinforcement Learning Framework for News Recommendation , 2018, WWW.

[81]  Harald Steck,et al.  Item popularity and recommendation accuracy , 2011, RecSys '11.

[82]  Minlie Huang,et al.  Topic-Guided Conversational Recommender in Multiple Domains , 2022, IEEE Transactions on Knowledge and Data Engineering.

[83]  Xin Wang,et al.  Interactive Social Recommendation , 2017, CIKM.

[84]  Craig Boutilier,et al.  A POMDP formulation of preference elicitation problems , 2002, AAAI/IAAI.

[85]  Min-Yen Kan,et al.  Sequicity: Simplifying Task-oriented Dialogue Systems with Single Sequence-to-Sequence Architectures , 2018, ACL.

[86]  Ga Wu,et al.  Deep Language-based Critiquing for Recommender Systems , 2019 .

[87]  Joelle Pineau,et al.  How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation , 2016, EMNLP.

[88]  Dongyan Zhao,et al.  Are Training Samples Correlated? Learning to Generate Dialogue Responses with Multiple References , 2019, ACL.

[89]  Mehrbakhsh Nilashi,et al.  Collaborative filtering recommender systems , 2013 .

[90]  Thierry Bertin-Mahieux,et al.  The Million Song Dataset , 2011, ISMIR.

[91]  Barry Smyth,et al.  Compound Critiques for Conversational Recommender Systems , 2004, IEEE/WIC/ACM International Conference on Web Intelligence (WI'04).

[92]  Larisa Shwartz,et al.  Online Interactive Collaborative Filtering Using Multi-Armed Bandit with Dependent Arms , 2017, IEEE Transactions on Knowledge and Data Engineering.

[93]  Lina Yao,et al.  Deep Learning Based Recommender System , 2017, ACM Comput. Surv..

[94]  Martha Larson,et al.  Collaborative Filtering beyond the User-Item Matrix , 2014, ACM Comput. Surv..

[95]  Jens Lehmann,et al.  DBpedia - A crystallization point for the Web of Data , 2009, J. Web Semant..

[96]  Hang Li,et al.  Convolutional Neural Network Architectures for Matching Natural Language Sentences , 2014, NIPS.

[97]  Patrick Gallinari,et al.  Ranking with non-random missing ratings: influence of popularity and positivity on evaluation metrics , 2012, RecSys.

[98]  Scott Sanner,et al.  Bayesian Preference Elicitation with Keyphrase-Item Coembeddings for Interactive Recommendation , 2021, UMAP.

[99]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[100]  Kun Gai,et al.  Learning Tree-based Deep Model for Recommender Systems , 2018, KDD.

[101]  Xing Xie,et al.  Towards Explainable Conversational Recommendation , 2020, IJCAI.

[102]  Arantxa Otegi,et al.  Survey on evaluation methods for dialogue systems , 2019, Artificial Intelligence Review.

[103]  Bowen Zhou,et al.  Improved Representation Learning for Question Answer Matching , 2016, ACL.

[104]  Scott Sanner,et al.  Deep Critiquing for VAE-based Recommender Systems , 2020, SIGIR.

[105]  Tie-Yan Liu,et al.  LightGBM: A Highly Efficient Gradient Boosting Decision Tree , 2017, NIPS.

[106]  Filip Radlinski,et al.  Towards Conversational Recommender Systems , 2016, KDD.

[107]  Jie Liu,et al.  Representing and Recommending Shopping Baskets with Complementarity, Compatibility and Loyalty , 2018, CIKM.

[108]  Izak Benbasat,et al.  Research Note - A Contingency Approach to Investigating the Effects of User-System Interaction Modes of Online Decision Aids , 2013, Inf. Syst. Res..

[109]  Soujanya Poria,et al.  Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering , 2021, ArXiv.

[110]  Xueqi Cheng,et al.  Text Matching as Image Recognition , 2016, AAAI.

[111]  P. McCullagh,et al.  Generalized Linear Models , 1972, Predictive Analytics.

[112]  Xu Jun,et al.  SIGIR 2014 workshop on semantic matching in information retrievalProceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval - SIGIR '14 , 2014 .

[113]  Paul Covington,et al.  Deep Neural Networks for YouTube Recommendations , 2016, RecSys.

[114]  Yann Dauphin,et al.  Deal or No Deal? End-to-End Learning of Negotiation Dialogues , 2017, EMNLP.

[115]  Mirella Lapata,et al.  Automatic Evaluation of Text Coherence: Models and Representations , 2005, IJCAI.

[116]  Walid Krichene,et al.  On Sampled Metrics for Item Recommendation , 2020, KDD.

[117]  Zoubin Ghahramani,et al.  Probabilistic Matrix Factorization with Non-random Missing Data , 2014, ICML.

[118]  M. de Rijke,et al.  A Survey of Query Auto Completion in Information Retrieval , 2016, Found. Trends Inf. Retr..

[119]  Wolfgang Minker,et al.  Emotion recognition and adaptation in spoken dialogue systems , 2010, Int. J. Speech Technol..

[120]  Yong Yu,et al.  Large-scale Interactive Recommendation with Tree-structured Policy Gradient , 2018, AAAI.

[121]  Xuanjing Huang,et al.  Convolutional Neural Tensor Network Architecture for Community-Based Question Answering , 2015, IJCAI.

[122]  A. Tversky,et al.  Context-dependent preferences , 1993 .

[123]  M. de Rijke,et al.  Query Resolution for Conversational Search with Limited Supervision , 2020, SIGIR.

[124]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[125]  M. de Rijke,et al.  Improving Response Quality with Backward Reasoning in Open-domain Dialogue Systems , 2021, SIGIR.

[126]  Bing Liu,et al.  User Memory Reasoning for Conversational Recommendation , 2020, COLING.

[127]  Stefan Ultes,et al.  MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling , 2018, EMNLP.

[128]  Flavian Vasile,et al.  BLOB: A Probabilistic Model for Recommendation that Combines Organic and Bandit Signals , 2020, KDD.

[129]  Thomas Nedelec,et al.  Offline A/B Testing for Recommender Systems , 2018, WSDM.

[130]  Stacy Marsella,et al.  A Socially-Aware Conversational Recommender System for Personalized Recipe Recommendations , 2020, HAI.

[131]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[132]  Hyunsouk Cho,et al.  MeLU: Meta-Learned User Preference Estimator for Cold-Start Recommendation , 2019, KDD.

[133]  Peter L. Bartlett,et al.  RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning , 2016, ArXiv.

[134]  Ed H. Chi,et al.  Top-K Off-Policy Correction for a REINFORCE Recommender System , 2018, WSDM.

[135]  Tat-Seng Chua,et al.  Knowledge-aware Multimodal Dialogue Systems , 2018, ACM Multimedia.

[136]  Wayne Xin Zhao,et al.  Towards Topic-Guided Conversational Recommender System , 2020, COLING.

[137]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[138]  Xiangnan He,et al.  Interactive Path Reasoning on Graph for Conversational Recommendation , 2020, KDD.

[139]  Mirella Lapata,et al.  Ranking Sentences for Extractive Summarization with Reinforcement Learning , 2018, NAACL.

[140]  Ruhi Sarikaya,et al.  Deep belief network based semantic taggers for spoken language understanding , 2013, INTERSPEECH.

[141]  Boi Faltings,et al.  Decision Tradeoff Using Example-Critiquing and Constraint Programming , 2004, Constraints.

[142]  M. de Rijke,et al.  Thinking Globally, Acting Locally: Distantly Supervised Global-to-Local Knowledge Selection for Background Based Conversation , 2019, AAAI.

[143]  Yan Feng,et al.  SamWalker: Social Recommendation with Informative Sampling Strategy , 2019, WWW.

[144]  Shuohang Wang,et al.  Learning Natural Language Inference with LSTM , 2015, NAACL.

[145]  Giuseppe Carenini,et al.  Towards more conversational and collaborative recommender systems , 2003, IUI '03.

[146]  Chris Callison-Burch,et al.  Comparison of Diverse Decoding Methods from Conditional Language Models , 2019, ACL.

[147]  Michael R. Lyu,et al.  Learning latent semantic relations from clickthrough data for query suggestion , 2008, CIKM '08.

[148]  Mark P. Graus,et al.  Improving the User Experience during Cold Start through Choice-Based Preference Elicitation , 2015, RecSys.

[149]  Peter Auer,et al.  Using Confidence Bounds for Exploitation-Exploration Trade-offs , 2003, J. Mach. Learn. Res..

[150]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[151]  Xiuqiang He,et al.  A General Knowledge Distillation Framework for Counterfactual Recommendation via Uniform Data , 2020, SIGIR.

[152]  Yejin Choi,et al.  Deep Communicating Agents for Abstractive Summarization , 2018, NAACL.

[153]  M. de Rijke,et al.  A Cooperative Memory Network for Personalized Task-oriented Dialogue Systems with Incomplete User Profiles , 2021, WWW.

[154]  Hang Li,et al.  A Deep Architecture for Matching Short Texts , 2013, NIPS.

[155]  Yuval Shahar,et al.  Utility Elicitation as a Classification Problem , 1998, UAI.

[156]  Guorui Zhou,et al.  Deep Interest Network for Click-Through Rate Prediction , 2017, KDD.

[157]  Yiqun Liu,et al.  Do users rate or review?: boost phrase-level sentiment labeling with review-level sentiment classification , 2014, SIGIR.

[158]  Christopher Joseph Pal,et al.  Towards Deep Conversational Recommendations , 2018, NeurIPS.

[159]  Guokun Lai,et al.  Explicit factor models for explainable recommendation based on phrase-level sentiment analysis , 2014, SIGIR.

[160]  M. de Rijke,et al.  When People Change their Mind: Off-Policy Evaluation in Non-stationary Recommendation Environments , 2019, WSDM.

[161]  Peter Sunehag,et al.  Deep Reinforcement Learning with Attention for Slate Markov Decision Processes with High-Dimensional States and Actions , 2015, ArXiv.

[162]  Zeb Kurth-Nelson,et al.  Learning to reinforcement learn , 2016, CogSci.

[163]  Jason Weston,et al.  Key-Value Memory Networks for Directly Reading Documents , 2016, EMNLP.

[164]  Dietmar Jannach,et al.  End-to-End Learning for Conversational Recommendation: A Long Way to Go? , 2020, IntRS@RecSys.

[165]  Shuyuan Xu,et al.  HOOPS: Human-in-the-Loop Graph Reasoning for Conversational Recommendation , 2021, SIGIR.

[166]  Tat-Seng Chua,et al.  Neural Factorization Machines for Sparse Predictive Analytics , 2017, SIGIR.

[167]  Bin Shen,et al.  Collaborative Memory Network for Recommendation Systems , 2018, SIGIR.

[168]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[169]  Yuanyuan Jin,et al.  Conversational Music Recommendation based on Bandits , 2020, 2020 IEEE International Conference on Knowledge Graph (ICKG).

[170]  Evangelos Kanoulas,et al.  Towards Question-based Recommender Systems , 2020, SIGIR.

[171]  Hsuan-Tien Lin,et al.  Pseudo-reward Algorithms for Contextual Bandits with Linear Payoff Functions , 2014, ACML.

[172]  Minlie Huang,et al.  A Pre-training Based Personalized Dialogue Generation Model with Persona-sparse Data , 2019, AAAI.

[173]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[174]  Ji-Rong Wen,et al.  CRSLab: An Open-Source Toolkit for Building Conversational Recommender System , 2021, ACL.

[175]  Qing Wang,et al.  Online Context-Aware Recommendation with Time Varying Multi-Armed Bandit , 2016, KDD.

[176]  Peng Pan,et al.  A Text-based Deep Reinforcement Learning Framework for Interactive Recommendation , 2020, ECAI.

[177]  Scott Sanner,et al.  Latent Linear Critiquing for Conversational Recommender Systems , 2020, WWW.

[178]  Michael N. Katehakis,et al.  The Multi-Armed Bandit Problem: Decomposition and Computation , 1987, Math. Oper. Res..

[179]  Tat-Seng Chua,et al.  Neural Multimodal Belief Tracker with Adaptive Attention for Dialogue Systems , 2019, WWW.

[180]  Dongyan Zhao,et al.  Meaningful Answer Generation of E-Commerce Question-Answering , 2020, ACM Trans. Inf. Syst..

[181]  Natasha Jaques,et al.  Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems , 2019, NeurIPS.

[182]  Kristian J. Hammond,et al.  The FindMe Approach to Assisted Browsing , 1997, IEEE Expert.

[183]  F. Mangili,et al.  A Bayesian Approach to Conversational Recommendation Systems , 2020, ArXiv.

[184]  Jiaxing Song,et al.  Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systems , 2019, KDD.

[185]  Pietro Liò,et al.  Deep Graph Infomax , 2018, ICLR.

[186]  Lihong Li,et al.  Toward Predicting the Outcome of an A/B Experiment for Search Relevance , 2015, WSDM.

[187]  Seungwhan Moon,et al.  OpenDialKG: Explainable Conversational Reasoning with Attention-based Walks over Knowledge Graphs , 2019, ACL.

[188]  A. Tordai,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017 .

[189]  Xu Chen,et al.  Towards Conversational Search and Recommendation: System Ask, User Respond , 2018, CIKM.

[190]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[191]  Richard S. Zemel,et al.  Collaborative Filtering and the Missing at Random Assumption , 2007, UAI.

[192]  M. de Rijke,et al.  Keeping Dataset Biases out of the Simulation: A Debiased Simulator for Reinforcement Learning based Recommender Systems , 2020, RecSys.

[193]  Xueqi Cheng,et al.  A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations , 2015, AAAI.

[194]  Francesco Ricci,et al.  Learning and adaptivity in interactive recommender systems , 2007, ICEC.

[195]  M. Mehdi Afsar,et al.  Reinforcement learning based recommender systems: A survey , 2021, ArXiv.

[196]  Liang Zhang,et al.  Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning , 2018, KDD.

[197]  Jianfeng Gao,et al.  Deep Reinforcement Learning for Dialogue Generation , 2016, EMNLP.

[198]  Konstantina Christakopoulou,et al.  Q&R: A Two-Stage Approach toward Interactive Recommendation , 2018, KDD.

[199]  David C. Atkins,et al.  Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approach , 2021, WWW.

[200]  Chen Cui,et al.  User Attention-guided Multimodal Dialog Systems , 2019, SIGIR.

[201]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[202]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[203]  Yehuda Koren,et al.  Modeling relationships at multiple scales to improve accuracy of large recommender systems , 2007, KDD '07.

[204]  Jure Leskovec,et al.  Inferring Networks of Substitutable and Complementary Products , 2015, KDD.

[205]  Yuan Qi,et al.  Generative Adversarial User Model for Reinforcement Learning Based Recommendation System , 2018, ICML.

[206]  Depeng Jin,et al.  Reinforced Negative Sampling for Recommendation with Exposure Data , 2019, IJCAI.

[207]  M. de Rijke,et al.  Conversations with Documents: An Exploration of Document-Centered Assistance , 2020, CHIIR.

[208]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[209]  Hongxia Yang,et al.  Towards Knowledge-Based Recommender Dialog System , 2019, EMNLP.

[210]  Weinan Zhang,et al.  Interactive Recommender System via Knowledge Graph-enhanced Reinforcement Learning , 2020, SIGIR.

[211]  Yi Zhang,et al.  Conversational Recommender System , 2018, SIGIR.

[212]  Masoud Mansoury,et al.  Multi-sided Exposure Bias in Recommendation , 2020, ArXiv.

[213]  S. Levine,et al.  Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems , 2020, ArXiv.

[214]  Richard Socher,et al.  Ask Me Anything: Dynamic Memory Networks for Natural Language Processing , 2015, ICML.

[215]  Jieping Ye,et al.  Hierarchical Adaptive Contextual Bandits for Resource Constraint based Recommendation , 2020, WWW.

[216]  Harald Steck,et al.  Evaluation of recommendations: rating-prediction and ranking , 2013, RecSys.

[217]  Austin Henderson,et al.  RABBIT: An Intelligent Database Assistant , 1982, AAAI.

[218]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[219]  Sungjin Lee,et al.  Jointly Optimizing Diversity and Relevance in Neural Response Generation , 2019, NAACL.

[220]  Craig Boutilier,et al.  Gradient-based Optimization for Bayesian Preference Elicitation , 2019, AAAI.

[221]  Wei Wang,et al.  Fast Adaptation for Cold-start Collaborative Filtering with Meta-learning , 2020, 2020 IEEE International Conference on Data Mining (ICDM).

[222]  Tat-Seng Chua,et al.  Denoising Implicit Feedback for Recommendation , 2020, WSDM.

[223]  Yoshua Bengio,et al.  Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding , 2013, INTERSPEECH.

[224]  Rui Yan,et al.  Learning to Respond with Deep Neural Networks for Retrieval-Based Human-Computer Conversation System , 2016, SIGIR.

[225]  Bowen Zhou,et al.  Applying deep learning to answer selection: A study and an open task , 2015, 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU).

[226]  Qiong Wu,et al.  Diversified Interactive Recommendation with Implicit Feedback , 2020, AAAI.

[227]  Steffen Rendle,et al.  Factorization Machines , 2010, 2010 IEEE International Conference on Data Mining.

[228]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[229]  Heng-Tze Cheng,et al.  Wide & Deep Learning for Recommender Systems , 2016, DLRS@RecSys.

[230]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[231]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[232]  Kun Zhou,et al.  Improving Conversational Recommender Systems via Knowledge Graph based Semantic Fusion , 2020, KDD.

[233]  Eduardo Sánchez Vila,et al.  Choice-Based Recommender Systems , 2016, RecTour@RecSys.

[234]  Tomohiro Takagi,et al.  Dialogue Based Recommender System that Flexibly Mixes Utterances and Recommendations , 2019, 2019 IEEE/WIC/ACM International Conference on Web Intelligence (WI).

[235]  Xiaoyan Zhu,et al.  Assigning Personality/Profile to a Chatting Machine for Coherent Conversation Generation , 2018, IJCAI.

[236]  Xiangnan He,et al.  Seamlessly Unifying Attributes and Items: Conversational Recommendation for Cold-Start Users , 2020, ArXiv.

[237]  Zhaochun Ren,et al.  Explicit State Tracking with Semi-Supervisionfor Neural Dialogue Generation , 2018, CIKM.

[238]  Pat Langley,et al.  A Personalized System for Conversational Recommendations , 2011, J. Artif. Intell. Res..

[239]  Karthik Ramani,et al.  Deconvolving Feedback Loops in Recommender Systems , 2016, NIPS.

[240]  Paul N. Bennett,et al.  Leading Conversational Search by Suggesting Useful Questions , 2020, WWW.

[241]  Deborah Estrin,et al.  Unbiased offline recommender evaluation for missing-not-at-random implicit feedback , 2018, RecSys.

[242]  David R. Traum,et al.  Evaluation Understudy for Dialogue Coherence Models , 2008, SIGDIAL Workshop.

[243]  Xiangnan He,et al.  Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems , 2020, WSDM.

[244]  Catherine Havasi,et al.  ConceptNet 5.5: An Open Multilingual Graph of General Knowledge , 2016, AAAI.

[245]  Tim Hussein,et al.  Choice-based preference elicitation for collaborative filtering recommender systems , 2014, CHI.

[246]  Jianfeng Gao,et al.  A Diversity-Promoting Objective Function for Neural Conversation Models , 2015, NAACL.

[247]  Chin-Yew Lin,et al.  ROUGE: A Package for Automatic Evaluation of Summaries , 2004, ACL 2004.

[248]  He Sun,et al.  Choice-Based Recommender Systems: A Unified Approach to Achieving Relevancy and Diversity , 2014, Oper. Res..

[249]  Anton van den Hengel,et al.  Image-Based Recommendations on Styles and Substitutes , 2015, SIGIR.

[250]  Yixin Cao,et al.  Reinforced Negative Sampling over Knowledge Graph for Recommendation , 2020, WWW.

[251]  Marco Degemmis,et al.  A Dataset of Real Dialogues for Conversational Recommender Systems , 2019, CLiC-it.

[252]  Xinya Du,et al.  Learning to Ask: Neural Question Generation for Reading Comprehension , 2017, ACL.

[253]  Xiaoyan Zhu,et al.  Contextual Combinatorial Bandit and its Application on Diversified Online Recommendation , 2014, SDM.

[254]  Gökhan Tür,et al.  Use of kernel deep convex networks and end-to-end learning for spoken language understanding , 2012, 2012 IEEE Spoken Language Technology Workshop (SLT).

[255]  Katrien Verbert,et al.  Interactive recommender systems: A survey of the state of the art and future research challenges and opportunities , 2016, Expert Syst. Appl..

[256]  Xiaoyan Zhu,et al.  Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory , 2017, AAAI.

[257]  Francesco Ricci,et al.  Feature selection methods for conversational recommender systems , 2005, 2005 IEEE International Conference on e-Technology, e-Commerce and e-Service.

[258]  Hui Ye,et al.  Agenda-Based User Simulation for Bootstrapping a POMDP Dialogue System , 2007, NAACL.

[259]  Shuai Yuan,et al.  BLOMA: Explain Collaborative Filtering via Boosted Local Rank-One Matrix Approximation , 2019, DASFAA.

[260]  Kilian Q. Weinberger,et al.  Simplifying Graph Convolutional Networks , 2019, ICML.

[261]  M. de Rijke,et al.  Simulating User Satisfaction for the Evaluation of Task-oriented Dialogue Systems , 2021, SIGIR.

[262]  Filip Radlinski,et al.  Preference elicitation as an optimization problem , 2018, RecSys.

[263]  Weiyan Shi,et al.  INSPIRED: Toward Sociable Recommendation Dialog Systems , 2020, EMNLP.

[264]  Le Wu,et al.  A Survey on Neural Recommendation: From Collaborative Filtering to Content and Context Enriched Recommendation , 2021, ArXiv.

[265]  Craig Boutilier,et al.  Reinforcement Learning for Slate-based Recommender Systems: A Tractable Decomposition and Practical Methodology , 2019, ArXiv.

[266]  Ji-Rong Wen,et al.  One Chatbot Per Person: Creating Personalized Chatbots based on Implicit User Profiles , 2021, SIGIR.

[267]  Maarten de Rijke,et al.  OpenSearch: Lessons Learned from an Online Evaluation Campaign , 2018, ACM J. Data Inf. Qual..

[268]  Yun-Nung Chen,et al.  QAInfomax: Learning Robust Question Answering System by Mutual Information Maximization , 2019, EMNLP.

[269]  P. Heng,et al.  Personalized Re-ranking with Item Relationships for E-commerce , 2020, CIKM.

[270]  Giovanni Semeraro,et al.  Conversational Recommender Systems and natural language: : A study through the ConveRSE framework , 2020, Decis. Support Syst..

[271]  Hanwang Zhang,et al.  "Click" Is Not Equal to "Like": Counterfactual Recommendation for Mitigating Clickbait Issue , 2020, ArXiv.