Pattern and content controlled response generation

Abstract Controllable response generation is an attractive and valuable task to the success of conversational systems. However, controlling both pattern and content of the response has not been well studied in existing models since they are mainly based on matching mechanisms. To tackle the problem, we first design a pattern model to automatically learn and extract speech patterns from words. The pattern is then integrated into the encoder–decoder model to control the response pattern. Second, a sentence sampling algorithm is built to directly insert or delete words in the generated response, so that the content is controlled. In this two-stage framework, the response could be explicitly controlled by the pattern and content, without any human annotation of the post-response dataset. Experiments show the proposed framework achieves better performance in response controllability than the state-of-the-art.

[1]  Thomas Holtgraves,et al.  Text messaging, personality, and the social context , 2011 .

[2]  Zhiting Hu,et al.  A Survey of Knowledge-enhanced Text Generation , 2020, ACM Comput. Surv..

[3]  Jianfeng Gao,et al.  A Diversity-Promoting Objective Function for Neural Conversation Models , 2015, NAACL.

[4]  Quoc V. Le,et al.  A Neural Conversational Model , 2015, ArXiv.

[5]  David Yarowsky,et al.  Classifying latent user attributes in twitter , 2010, SMUC '10.

[6]  Steven Bird,et al.  NLTK: The Natural Language Toolkit , 2002, ACL.

[7]  Steve J. Young,et al.  Partially observable Markov decision processes for spoken dialog systems , 2007, Comput. Speech Lang..

[8]  Derek Ruths,et al.  Gender Inference of Twitter Users in Non-English Contexts , 2013, EMNLP.

[9]  Jun Zhao,et al.  Generating Natural Answers by Incorporating Copying and Retrieving Mechanisms in Sequence-to-Sequence Learning , 2017, ACL.

[10]  Pan Zhou,et al.  Emotion-aware Chat Machine: Automatic Emotional Response Generation for Human-like Emotional Interaction , 2019, CIKM.

[11]  Kevin Gimpel,et al.  A Multi-Task Approach for Disentangling Syntax and Semantics in Sentence Representations , 2019, NAACL.

[12]  Niranjan Balasubramanian,et al.  Human Centered NLP with User-Factor Adaptation , 2017, EMNLP.

[13]  J. M. Digman PERSONALITY STRUCTURE: EMERGENCE OF THE FIVE-FACTOR MODEL , 1990 .

[14]  Xiaoyan Zhu,et al.  Assigning personality/identity to a chatting machine for coherent conversation generation , 2017, ArXiv.

[15]  David Konopnicki,et al.  Neural Response Generation for Customer Service based on Personality Traits , 2017, INLG.

[16]  Piji Li,et al.  Persona-Aware Tips Generation? , 2019, WWW.

[17]  Pascale Fung,et al.  Personalizing Dialogue Agents via Meta-Learning , 2019, ACL.

[18]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[19]  Kallirroi Georgila,et al.  Reinforcement Learning of Question-Answering Dialogue Policies for Virtual Museum Guides , 2012, SIGDIAL Conference.

[20]  Sungjin Lee,et al.  Structuring Latent Spaces for Stylized Response Generation , 2019, EMNLP.

[21]  Jindrich Libovický,et al.  Attention Strategies for Multi-Source Sequence-to-Sequence Learning , 2017, ACL.

[22]  Wei-Ying Ma,et al.  Topic Aware Neural Response Generation , 2016, AAAI.

[23]  Zhe Gan,et al.  POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training , 2020, EMNLP.

[24]  C. Tappert,et al.  A Survey of Binary Similarity and Distance Measures , 2010 .

[25]  Song Liu,et al.  Personalized Dialogue Generation with Diversified Traits , 2019, ArXiv.

[26]  Jiajun Zhang,et al.  Keywords-Guided Abstractive Sentence Summarization , 2020, AAAI.

[27]  Hang Li,et al.  An Information Retrieval Approach to Short Text Conversation , 2014, ArXiv.

[28]  Lei Li,et al.  CGMH: Constrained Sentence Generation by Metropolis-Hastings Sampling , 2018, AAAI.

[29]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[30]  Jason Yosinski,et al.  Plug and Play Language Models: A Simple Approach to Controlled Text Generation , 2020, ICLR.

[31]  Wei Wu,et al.  Knowledge-Grounded Dialogue Generation with Pre-trained Language Models , 2020, EMNLP.

[32]  Xiaoyan Zhu,et al.  Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory , 2017, AAAI.

[33]  Hua Wu,et al.  PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable , 2020, ACL.

[34]  Milica Gasic,et al.  POMDP-Based Statistical Spoken Dialog Systems: A Review , 2013, Proceedings of the IEEE.

[35]  Piji Li,et al.  A Neural Topical Expansion Framework for Unstructured Persona-oriented Dialogue Generation , 2020, ECAI.

[36]  Hang Li,et al.  Neural Responding Machine for Short-Text Conversation , 2015, ACL.

[37]  Qun Liu,et al.  Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search , 2017, ACL.

[38]  Lei Zheng,et al.  Texygen: A Benchmarking Platform for Text Generation Models , 2018, SIGIR.

[39]  Shuming Shi,et al.  Rigid Formats Controlled Text Generation , 2020, ACL.

[40]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[41]  Joelle Pineau,et al.  Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models , 2015, AAAI.

[42]  Steven Skiena,et al.  Latent human traits in the language of social media: An open-vocabulary approach , 2017, PloS one.

[43]  Jakob Uszkoreit,et al.  Insertion Transformer: Flexible Sequence Generation via Insertion Operations , 2019, ICML.

[44]  Gregory J. Park,et al.  Automatic personality assessment through social media language. , 2015, Journal of personality and social psychology.

[45]  Tal Yarkoni Personality in 100,000 Words: A large-scale analysis of personality and word use among bloggers. , 2010, Journal of research in personality.

[46]  Heng Ji,et al.  A Novel Neural Topic Model and Its Supervised Extension , 2015, AAAI.

[47]  William Yang Wang,et al.  MojiTalk: Generating Emotional Responses at Scale , 2017, ACL.

[48]  Margaret L. Kern,et al.  Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach , 2013, PloS one.

[49]  S. Bookheimer,et al.  Form and Content Dissociating Syntax and Semantics in Sentence Comprehension , 1999, Neuron.