Topic-Oriented Spoken Dialogue Summarization for Customer Service with Saliency-Aware Topic Modeling

In a customer service system, dialogue summarization can boost service efficiency by automatically creating summaries for long spoken dialogues in which customers and agents try to address issues about specific topics. In this work, we focus on topic-oriented dialogue summarization, which generates highly abstractive summaries that preserve the main ideas from dialogues. In spoken dialogues, abundant dialogue noise and common semantics could obscure the underlying informative content, making the general topic modeling approaches difficult to apply. In addition, for customer service, role-specific information matters and is an indispensable part of a summary. To effectively perform topic modeling on dialogues and capture multi-role information, in this work we propose a novel topic-augmented two-stage dialogue summarizer (TDS) jointly with a saliency-aware neural topic model (SATM) for topic-oriented summarization of customer service dialogues. Comprehensive studies on a real-world Chinese customer service dataset demonstrated the superiority of our method against several strong baselines.

[1]  Bowen Zhou,et al.  Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond , 2016, CoNLL.

[2]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[3]  Nancy F. Chen,et al.  Topic-Aware Pointer-Generator Networks for Summarizing Spoken Conversations , 2019, 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU).

[4]  Jieping Ye,et al.  Automatic Dialogue Summary Generation for Customer Service , 2019, KDD.

[5]  Phil Blunsom,et al.  Discovering Discrete Latent Topics with Neural Variational Inference , 2017, ICML.

[6]  Navdeep Jaitly,et al.  Pointer Networks , 2015, NIPS.

[7]  Yan Liu,et al.  Dial2Desc: End-to-end Dialogue Description Generation , 2018, ArXiv.

[8]  Jihong Ouyang,et al.  Dirichlet Multinomial Mixture with Variational Manifold Regularization: Topic Modeling over Short Texts , 2019, AAAI.

[9]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[10]  Claire Cardie,et al.  Unsupervised Topic Modeling Approaches to Decision Summarization in Spoken Meetings , 2012, SIGDIAL Conference.

[11]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[12]  Michalis Vazirgiannis,et al.  Combining Graph Degeneracy and Submodularity for Unsupervised Extractive Summarization , 2017, NFiS@EMNLP.

[13]  Yen-Chun Chen,et al.  Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting , 2018, ACL.

[14]  Hui Lin,et al.  Evaluating the effectiveness of features and sampling in extractive meeting summarization , 2008, 2008 IEEE Spoken Language Technology Workshop.

[15]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[16]  Satoshi Takahashi,et al.  Learning to Model Domain-Specific Utterance Sequences for Extractive Summarization of Contact Center Dialogues , 2010, COLING.

[17]  Christopher D. Manning,et al.  Get To The Point: Summarization with Pointer-Generator Networks , 2017, ACL.

[18]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[19]  Mirella Lapata,et al.  Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization , 2018, EMNLP.

[20]  Dilek Z. Hakkani-Tür,et al.  A global optimization framework for meeting summarization , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[21]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[22]  Owen Rambow,et al.  Summarizing Email Threads , 2004, NAACL.

[23]  Shiliang Zhang,et al.  Investigation of Modeling Units for Mandarin Speech Recognition Using Dfsmn-ctc-smbr , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[24]  Chin-Yew Lin,et al.  ROUGE: A Package for Automatic Evaluation of Summaries , 2004, ACL 2004.

[25]  Yue Zhang,et al.  Contrastive Attention Mechanism for Abstractive Sentence Summarization , 2019, EMNLP.

[26]  Klaus Zechner,et al.  Automatic generation of concise summaries of spoken dialogues in unrestricted domains , 2001, SIGIR '01.

[27]  Mirella Lapata,et al.  Text Summarization with Pretrained Encoders , 2019, EMNLP.

[28]  Yue Wang,et al.  Filtering out the noise in short text topic modeling , 2018, Inf. Sci..

[29]  Giuseppe Carenini,et al.  Abstractive Meeting Summarization with Entailment and Fusion , 2013, ENLG.

[30]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[31]  Jean-Pierre Lorré,et al.  Unsupervised Abstractive Meeting Summarization with Multi-Sentence Compression and Budgeted Submodular Maximization , 2018, ACL.

[32]  Giuseppe Carenini,et al.  A Template-based Abstractive Meeting Summarization: Leveraging Summary and Source Text Relationships , 2014, INLG.

[33]  Ming Zhou,et al.  HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization , 2019, ACL.

[34]  Zhenglu Yang,et al.  Document Summarization with VHTM: Variational Hierarchical Topic-Aware Mechanism , 2020, AAAI.

[35]  Guodong Zhou,et al.  Sentiment Classification in Customer Service Dialogue with Topic-Aware Multi-Task Learning , 2020, AAAI.

[36]  Xin Zhou,et al.  Legal Summarization for Multi-role Debate Dialogue via Controversy Focus Mining and Multi-task Learning , 2019, CIKM.

[37]  Thomas Hofmann,et al.  Probabilistic Latent Semantic Indexing , 1999, SIGIR Forum.

[38]  Heng Ji,et al.  Keep Meeting Summaries on Topic: Abstractive Multi-Modal Meeting Summarization , 2019, ACL.

[39]  Li Wang,et al.  A Reinforced Topic-Aware Convolutional Sequence-to-Sequence Model for Abstractive Text Summarization , 2018, IJCAI.

[40]  Bowen Zhou,et al.  SummaRuNNer: A Recurrent Neural Network Based Sequence Model for Extractive Summarization of Documents , 2016, AAAI.

[41]  Vasudeva Varma,et al.  Topic-Focused Summarization of Chat Conversations , 2013, ECIR.