Smoothing Dialogue States for Open Conversational Machine Reading

Conversational machine reading (CMR) requires machines to communicate with humans through multi-turn interactions between two salient dialogue states of decision making and question generation processes. In open CMR settings, as the more realistic scenario, the retrieved background knowledge would be noisy, which results in severe challenges in the information transmission. Existing studies commonly train independent or pipeline systems for the two subtasks. However, those methods are trivial by using hard-label decisions to activate question generation, which eventually hinders the model performance. In this work, we propose an effective gating strategy by smoothing the two dialogue states in only one decoder and bridge decision making and question generation to provide a richer dialogue state reference. Experiments on the OR-ShARC dataset show the effectiveness of our method, which achieves new state-of-the-art results.

[1]  Minlie Huang,et al.  A Deep Sequential Model for Discourse Parsing on Multi-Party Dialogues , 2018, AAAI.

[2]  Jianfeng Gao,et al.  Challenges in Building Intelligent Open-domain Dialog Systems , 2019, ACM Trans. Inf. Syst..

[3]  Michael R. Lyu,et al.  Open-Retrieval Conversational Machine Reading , 2021, ArXiv.

[4]  Danqi Chen,et al.  Dense Passage Retrieval for Open-Domain Question Answering , 2020, EMNLP.

[5]  Mathias Niepert,et al.  Attending to Future Tokens for Bidirectional Sequence Generation , 2019, EMNLP.

[6]  Eunsol Choi,et al.  QuAC: Question Answering in Context , 2018, EMNLP.

[7]  Jianfeng Gao,et al.  DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation , 2020, ACL.

[8]  P. J. Price,et al.  Evaluation of Spoken Language Systems: the ATIS Domain , 1990, HLT.

[9]  Luke Zettlemoyer,et al.  E3: Entailment-driven Extracting and Editing for Conversational Machine Reading , 2019, ACL.

[10]  Xiaodong Liu,et al.  Unified Language Model Pre-training for Natural Language Understanding and Generation , 2019, NeurIPS.

[11]  Shafiq R. Joty,et al.  Explicit Memory Tracker with Coarse-to-Fine Reasoning for Conversational Machine Reading , 2020, ACL.

[12]  Claire Cardie,et al.  DREAM: A Challenge Data Set and Models for Dialogue-Based Reading Comprehension , 2019, TACL.

[13]  Yue Zhang,et al.  MuTual: A Dataset for Multi-Turn Dialogue Reasoning , 2020, ACL.

[14]  Lu Chen,et al.  Towards Universal Dialogue State Tracking , 2018, EMNLP.

[15]  Libo Qin,et al.  A Survey on Spoken Language Understanding: Recent Advances and New Frontiers , 2021, IJCAI.

[16]  Nicholas Asher,et al.  Discourse Structure and Dialogue Acts in Multiparty Dialogue: the STAC Corpus , 2016, LREC.

[17]  Hai Zhao,et al.  Dialogue Graph Modeling for Conversational Machine Reading , 2020, FINDINGS.

[18]  Colin Raffel,et al.  Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer , 2019, J. Mach. Learn. Res..

[19]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[20]  James R. Glass,et al.  Asgard: A portable architecture for multilingual dialogue systems , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[21]  Xudong Luo,et al.  A Survey of Response Generation of Dialogue Systems , 2020 .

[22]  Danqi Chen,et al.  CoQA: A Conversational Question Answering Challenge , 2018, TACL.

[23]  Hai Zhao,et al.  Modeling Multi-turn Conversation with Deep Utterance Aggregation , 2018, COLING.

[24]  W. Bruce Croft,et al.  Open-Retrieval Conversational Question Answering , 2020, SIGIR.

[25]  Jian Zhang,et al.  SQuAD: 100,000+ Questions for Machine Comprehension of Text , 2016, EMNLP.

[26]  Xiaodong Zhang,et al.  Graph LSTM with Context-Gated Mechanism for Spoken Language Understanding , 2020, AAAI.

[27]  Percy Liang,et al.  Know What You Don’t Know: Unanswerable Questions for SQuAD , 2018, ACL.

[28]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[29]  Quan Z. Sheng,et al.  A Short Survey of Pre-trained Language Models for Conversational AI-A New Age in NLP , 2020, ACSW.

[30]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[31]  Hai Zhao,et al.  Lingke: a Fine-grained Multi-turn Chatbot for Customer Service , 2018, COLING.

[32]  Gokhan Tur,et al.  Spoken Language Understanding: Systems for Extracting Semantic Information from Speech , 2011 .

[33]  William C. Mann,et al.  Rhetorical Structure Theory: Toward a functional theory of text organization , 1988 .

[34]  Kang Min Yoo,et al.  DialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterances , 2020, AAAI.

[35]  Shuming Shi,et al.  Skeleton-to-Response: Dialogue Generation Guided by Retrieval Memory , 2018, NAACL.

[36]  Zhoujun Li,et al.  Neural Response Generation with Dynamic Vocabularies , 2017, AAAI.

[37]  Haejun Lee,et al.  SLM: Learning a Discourse Language Representation with Sentence Unshuffling , 2020, EMNLP.

[38]  Shafiq R. Joty,et al.  Discern: Discourse-Aware Entailment Reasoning Network for Conversational Machine Reading , 2020, EMNLP.

[39]  Guillaume Bouchard,et al.  Interpretation of Natural Language Rules in Conversational Machine Reading , 2018, EMNLP.

[40]  Hai Zhao,et al.  Retrospective Reader for Machine Reading Comprehension , 2020, AAAI.

[41]  Hai Zhao,et al.  Semantics-aware BERT for Language Understanding , 2020, AAAI.

[42]  Zhen-Hua Ling,et al.  MPC-BERT: A Pre-Trained Language Model for Multi-Party Conversation Understanding , 2021, ACL.