Neural Response Ranking for Social Conversation: A Data-Efficient Approach

The overall objective of ‘social’ dialogue systems is to support engaging, entertaining, and lengthy conversations on a wide variety of topics, including social chit-chat. Apart from raw dialogue data, user-provided ratings are the most common signal used to train such systems to produce engaging responses. In this paper we show that social dialogue systems can be trained effectively from raw unannotated data. Using a dataset of real conversations collected in the 2017 Alexa Prize challenge, we developed a neural ranker for selecting ‘good’ system responses to user utterances, i.e. responses which are likely to lead to long and engaging conversations. We show that (1) our neural ranker consistently outperforms several strong baselines when trained to optimise for user ratings; (2) when trained on larger amounts of data and only using conversation length as the objective, the ranker performs better than the one trained using ratings – ultimately reaching a Precision@1 of 0.87. This advance will make data collection for social conversational agents simpler and less expensive in the future.

[1]  John Langford,et al.  A reliable effective terascale linear learning system , 2011, J. Mach. Learn. Res..

[2]  Zhou Yu,et al.  TickTock: A Non-Goal-Oriented Multimodal Dialog System with Engagement Awareness , 2015, AAAI Spring Symposia.

[3]  Gregory N. Hullender,et al.  Learning to rank using gradient descent , 2005, ICML.

[4]  Christopher D. Manning,et al.  Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling , 2005, ACL.

[5]  Denis G. Fedorenko,et al.  Avoiding Echo-Responses in a Retrieval-Based Conversation System , 2017, ArXiv.

[6]  Eric Gilbert,et al.  VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text , 2014, ICWSM.

[7]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[8]  Jeff Johnson,et al.  Billion-Scale Similarity Search with GPUs , 2017, IEEE Transactions on Big Data.

[9]  Milica Gasic,et al.  The Hidden Information State model: A practical framework for POMDP-based spoken dialogue management , 2010, Comput. Speech Lang..

[10]  Oliver Lemon,et al.  An Ensemble Model with Ranking for Social Dialogue , 2017, NIPS 2017.

[11]  Yichao Lu,et al.  A practical approach to dialogue response generation in closed domains , 2017, ArXiv.

[12]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[13]  Bowen Wu,et al.  Ranking Responses Oriented to Conversational Relevance in Chat-bots , 2016, COLING.

[14]  Javier Snaider,et al.  Conversational Contextual Cues: The Case of Personalization and History for Response Ranking , 2016, ArXiv.

[15]  Hoang Nguyen,et al.  Question-Answer Selection in User to User Marketplace Conversations , 2018, IWSDS.

[16]  Joelle Pineau,et al.  The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems , 2015, SIGDIAL Conference.

[17]  Alexander I. Rudnicky,et al.  RubyStar: A Non-Task-Oriented Mixture Model Dialog System , 2017, ArXiv.

[18]  Hang Li,et al.  An Information Retrieval Approach to Short Text Conversation , 2014, ArXiv.

[19]  Rui Yan,et al.  Learning to Respond with Deep Neural Networks for Retrieval-Based Human-Computer Conversation System , 2016, SIGIR.

[20]  Zhou Yu,et al.  Strategy and Policy Learning for Non-Task-Oriented Conversational Systems , 2016, SIGDIAL Conference.

[21]  Bonnie L. Webber,et al.  Edina: Building an Open Domain Socialbot with Self-dialogues , 2017, ArXiv.

[22]  Jianfeng Gao,et al.  Deep Reinforcement Learning for Dialogue Generation , 2016, EMNLP.

[23]  Xiang Li,et al.  Two are Better than One: An Ensemble of Retrieval- and Generation-Based Dialog Systems , 2016, ArXiv.

[24]  Quoc V. Le,et al.  A Neural Conversational Model , 2015, ArXiv.

[25]  Han Zhang,et al.  An Ensemble Approach to Conversation Generation , 2017, NLPCC.

[26]  Joelle Pineau,et al.  A Deep Reinforcement Learning Chatbot , 2017, ArXiv.

[27]  Oliver Lemon,et al.  Alana: Social Dialogue using an Ensemble Model and a Ranker trained on User Feedback , 2017 .

[28]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[29]  Peng Zhang,et al.  IRGAN: A Minimax Game for Unifying Generative and Discriminative Information Retrieval Models , 2017, SIGIR.

[30]  Rahul Goel,et al.  On Evaluating and Comparing Conversational Agents , 2018, ArXiv.

[31]  Francis R. Bach,et al.  Online Learning for Latent Dirichlet Allocation , 2010, NIPS.