A Scalable Framework for Learning From Implicit User Feedback to Improve Natural Language Understanding in Large-Scale Conversational AI Systems

Natural Language Understanding (NLU) is an established component within a conversational AI or digital assistant system, and it is responsible for producing semantic understanding of a user request. We propose a scalable and automatic approach for improving NLU in a large-scale conversational AI system by leveraging implicit user feedback, with an insight that user interaction data and dialog context have rich information embedded from which user satisfaction and intention can be inferred. In particular, we propose a general domain-agnostic framework for curating new supervision data for improving NLU from live production traffic. With an extensive set of experiments, we show the results of applying the framework and improving NLU for a large-scale production system and show its impact across 10 domains.

[1]  Gökhan Tür,et al.  Multi-Domain Joint Semantic Frame Parsing Using Bi-Directional RNN-LSTM , 2016, INTERSPEECH.

[2]  Alvin Cheung,et al.  Learning a Neural Semantic Parser from User Feedback , 2017, ACL.

[3]  Shahram Khadivi,et al.  Learning from Chunk-based Feedback in Neural Machine Translation , 2018, ACL.

[4]  Bing Liu,et al.  Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling , 2016, INTERSPEECH.

[5]  Zhiguo Wang,et al.  Bilateral Multi-Perspective Matching for Natural Language Sentences , 2017, IJCAI.

[6]  Jürgen Schmidhuber,et al.  Highway Networks , 2015, ArXiv.

[7]  Julian J. McAuley,et al.  VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback , 2015, AAAI.

[8]  Jakob Uszkoreit,et al.  Neural Paraphrase Identification of Questions with Noisy Pretraining , 2017, SWCN@EMNLP.

[9]  Weinan Zhang,et al.  Exploring Implicit Feedback for Open Domain Conversation Generation , 2018, AAAI.

[10]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[11]  Cícero Nogueira dos Santos,et al.  Learning Hybrid Representations to Retrieve Semantically Equivalent Questions , 2015, ACL.

[12]  Gökhan Tür,et al.  Extending domain coverage of language understanding systems via intent transfer between domains using knowledge graphs and search query click logs , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Xuehua Shen,et al.  Context-sensitive information retrieval using implicit feedback , 2005, SIGIR '05.

[14]  Rahul Gupta,et al.  A Re-Ranker Scheme For Integrating Large Scale NLU Models , 2018, 2018 IEEE Spoken Language Technology Workshop (SLT).

[15]  W. Bruce Croft,et al.  Leverage Implicit Feedback for Context-aware Product Search , 2019, eCOM@SIGIR.

[16]  Ruhi Sarikaya.  An overview of the system architecture and key components The Technology Behind Personal Digital Assistants , 2022 .

[17]  Yifan Hu,et al.  Collaborative Filtering for Implicit Feedback Datasets , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[18]  Xiao Yang,et al.  Leveraging User Engagement Signals For Entity Labeling in a Virtual Assistant , 2019, ArXiv.

[19]  Lihong Li,et al.  Neural Approaches to Conversational AI , 2019, Found. Trends Inf. Retr..

[20]  Martha Larson,et al.  Factorization Machines for Data with Implicit Feedback , 2018, ArXiv.

[21]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[22]  Jiahui Liu,et al.  Personalized news recommendation based on click behavior , 2010, IUI '10.

[23]  Haoyu Wang,et al.  Adversarial Binary Collaborative Filtering for Implicit Feedback , 2019, AAAI.

[24]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[25]  Masatoshi Yoshikawa,et al.  Adaptive web search based on user profile constructed without any effort from users , 2004, WWW '04.

[26]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[27]  Veton Kepuska,et al.  Next-generation of virtual personal assistants (Microsoft Cortana, Apple Siri, Amazon Alexa and Google Home) , 2018, 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC).

[28]  Ruhi Sarikaya,et al.  Feedback-Based Self-Learning in Large-Scale Conversational AI Agents , 2019, AAAI.

[29]  Tat-Seng Chua,et al.  Fast Matrix Factorization for Online Recommendation with Implicit Feedback , 2016, SIGIR.

[30]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.