Content-Oriented User Modeling for Personalized Response Ranking in Chatbots

Automatic chatbots (also known as chat-agents) have attracted much attention from both researching and industrial fields. Generally, the semantic relevance between users’ queries and the corresponding responses is considered as the essential element for conversation modeling in both generation and ranking based chat systems. By contrast, it is a nontrivial task to adopt the users’ information, such as preference, social role, etc., into conversational models reasonably, while users’ profiles play a significant role in the procedure of conversations by providing the implicit contexts. This paper aims to address the personalized response ranking task by incorporating user profiles into the conversation model. In our approach, users’ personalized representations are latently learned from the contents posted by them via a two-branch neural network. After that, a deep neural network architecture is further presented to learn the fusion representation of posts, responses, and personal information. In this way, the proposed model could understand conversations from the users’ perspective; hence, the more appropriate responses are selected for a specified person. The experimental results on two datasets from social network services demonstrate that our approach is hopeful to represent users’ personal information implicitly based on user generated contents, and it is promising to perform as an important component in chatbots to select the personalized responses for each user.

[1]  Hang Li,et al.  Neural Responding Machine for Short-Text Conversation , 2015, ACL.

[2]  Michal Barla Towards Social-based User Modeling and Personalization , 2010 .

[3]  Joelle Pineau,et al.  Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models , 2015, AAAI.

[4]  Xiang Li,et al.  StalemateBreaker: A Proactive Content-Introducing Approach to Automatic Human-Computer Conversation , 2016, IJCAI.

[5]  Joseph Weizenbaum,et al.  and Machine , 1977 .

[6]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[7]  Rabab Kreidieh Ward,et al.  Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval , 2015, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[8]  Ricardo Ribeiro,et al.  The Influence of Context on Dialogue Act Recognition , 2015, ArXiv.

[9]  Ryuichiro Higashinaka,et al.  User information extraction for personalized dialogue systems , 2016 .

[10]  Kenneth Mark Colby,et al.  Turing-like Indistinguishability Tests for the Calidation of a Computer Simulation of Paranoid Processes , 1972, Artif. Intell..

[11]  Gary Geunbae Lee,et al.  Acquisition and Use of Long-Term Memory for Personalized Dialog Systems , 2014, MA3HMI@INTERSPEECH.

[12]  Jianfeng Gao,et al.  A Persona-Based Neural Conversation Model , 2016, ACL.

[13]  Margaret L. Kern,et al.  Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach , 2013, PloS one.

[14]  Bowen Zhou,et al.  Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation , 2016, AAAI.

[15]  Rudolf Kadlec,et al.  Improved Deep Learning Baselines for Ubuntu Corpus Dialogs , 2015, ArXiv.

[16]  Hermann Ney,et al.  From Feedforward to Recurrent LSTM Neural Networks for Language Modeling , 2015, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[17]  Joakim Nivre,et al.  On the Semantics and Pragmatics of Linguistic Feedback , 1992, J. Semant..

[18]  Alan Ritter,et al.  Data-Driven Response Generation in Social Media , 2011, EMNLP.

[19]  Bowen Wu,et al.  Ranking Responses Oriented to Conversational Relevance in Chat-bots , 2016, COLING.

[20]  Luísa Coheur,et al.  Luke, I am Your Father: Dealing with Out-of-Domain Requests by Using Movies Subtitles , 2014, IVA.

[21]  Tetsunori Kobayashi,et al.  Automatic Expressive Opinion Sentence Generation for Enjoyable Conversational Systems , 2015, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[22]  Haizhou Li,et al.  IRIS: a Chat-oriented Dialogue System based on the Vector Space Model , 2012, ACL.

[23]  Joelle Pineau,et al.  The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems , 2015, SIGDIAL Conference.

[24]  Ahmad Abdel-Hafez,et al.  A Survey of User Modelling in Social Media Websites , 2013, Comput. Inf. Sci..

[25]  Dong Nguyen,et al.  Why Gender and Age Prediction from Tweets is Hard: Lessons from a Crowdsourcing Experiment , 2014, COLING.

[26]  Jianfeng Gao,et al.  A Neural Network Approach to Context-Sensitive Generation of Conversational Responses , 2015, NAACL.

[27]  Yuefeng Li,et al.  The state-of-the-art in personalized recommender systems for social networking , 2012, Artificial Intelligence Review.

[28]  Andrew Y. Ng,et al.  Semantic Compositionality through Recursive Matrix-Vector Spaces , 2012, EMNLP.

[29]  Jerry Feldman Mind as Machine: A History of Cognitive Science, Margaret Boden. Oxford U. Press (2006) , 2007 .

[30]  Ying Liu,et al.  Evaluating mass knowledge acquisition using the ALICE chatterbot: The AZ-ALICE dialog system , 2006, Int. J. Hum. Comput. Stud..

[31]  Alexander J. Smola,et al.  Scalable distributed inference of dynamic user interests for behavioral targeting , 2011, KDD.

[32]  Maryam Habibi,et al.  Keyword Extraction and Clustering for Document Recommendation in Conversations , 2015, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[33]  Sina Jafarpour,et al.  Filter, Rank, and Transfer the Knowledge: Learning to Chat , 2010 .

[34]  Sanja Fidler,et al.  Skip-Thought Vectors , 2015, NIPS.

[35]  Harry Bunt,et al.  Context and Dialogue Control , 1994 .

[36]  Kee-Sung Lee,et al.  Collaborative user modeling for enhanced content filtering in recommender systems , 2011, Decis. Support Syst..

[37]  Hang Li,et al.  An Information Retrieval Approach to Short Text Conversation , 2014, ArXiv.

[38]  David Vandyke,et al.  Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems , 2015, EMNLP.

[39]  Jianfeng Gao,et al.  A Diversity-Promoting Objective Function for Neural Conversation Models , 2015, NAACL.

[40]  A. M. Turing,et al.  Computing Machinery and Intelligence , 1950, The Philosophy of Artificial Intelligence.

[41]  Abhay Prakash,et al.  Emulating Human Conversations using Convolutional Neural Network-based IR , 2016, ArXiv.

[42]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[43]  Nicholas J. Belkin,et al.  Some(what) grand challenges for information retrieval , 2008, SIGF.

[44]  Sangdo Han,et al.  Open-domain personalized dialog system using user-interested topics in system responses , 2015, 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU).

[45]  Phil Blunsom,et al.  Teaching Machines to Read and Comprehend , 2015, NIPS.

[46]  Chunliang Lu,et al.  User Modeling and Tweets Recommendation Based on Wikipedia Concept Graph ∗ , 2012 .

[47]  Lin Sun,et al.  Modeling Semantic Relevance for Question-Answer Pairs in Web Social Communities , 2010, ACL.

[48]  Quoc V. Le,et al.  A Neural Conversational Model , 2015, ArXiv.