A Mutually Auxiliary Multitask Model With Self-Distillation for Emotion-Cause Pair Extraction

Emotion-cause pair extraction (ECPE), which aims to extract emotions and the corresponding causes in documents, has a wide range of applications in network public opinion analysis. Current two-stage methods first extract emotion and cause clauses, and then pair them. However, there are two problems in these methods: 1) the unidirectional enhancement between emotion and cause extraction fails to make full use of the correlation between them; 2) the errors from the first stage directly degrade the performance of the second stage. To address these problems, we firstly propose a mutually auxiliary multitask model to promote the extraction of emotion and cause clauses by adding two auxiliary tasks which are identical to the original tasks. The proposed model uses the predicted results generated by the two auxiliary tasks as extra features of each other’s main tasks, so as to establish the bidirectional correlation between emotion and cause extraction. Secondly, to reduce the influence of error propagation on the second stage, we design a self-distillation method for pairwise tasks to train the proposed model, which further improve the accuracy of emotion and cause extraction. Experimental results on the ECPE benchmark dataset show that the proposed model has achieved good performance on emotion-cause pair extraction, outperforming the baseline models by 1.92% in F1 score.

[1]  Hua Xu,et al.  A rule-based approach to emotion cause detection for Chinese micro-blogs , 2015, Expert Syst. Appl..

[2]  Daling Wang,et al.  A Co-Attention Neural Network Model for Emotion Cause Analysis with Emotional Context Awareness , 2018, EMNLP.

[3]  Zhang Xiong,et al.  Multiple Level Hierarchical Network-Based Clause Selection for Emotion Cause Extraction , 2019, IEEE Access.

[4]  Aytug Onan,et al.  A multiobjective weighted voting ensemble classifier based on differential evolution algorithm for text sentiment classification , 2016, Expert Syst. Appl..

[5]  Jianfei Yu,et al.  ECPE-2D: Emotion-Cause Pair Extraction based on Joint Two-Dimensional Representation, Interaction and Prediction , 2020, ACL.

[6]  Yongfeng Huang,et al.  A Multi-Task Learning Neural Network for Emotion-Cause Pair Extraction , 2020, ECAI.

[7]  Akshi Kumar,et al.  Multimedia Social Big Data: Mining , 2019, Intelligent Systems Reference Library.

[8]  Aytuğ Onan,et al.  Weighted word embeddings and clustering‐based identification of question topics in MOOC discussion forum posts , 2020, Comput. Appl. Eng. Educ..

[9]  Chu-Ren Huang,et al.  Emotion Cause Detection with Linguistic Constructions , 2010, COLING.

[10]  Yi Zhao,et al.  Emotion-Cause Joint Detection: A Unified Network with Dual Interaction for Emotion Cause Analysis , 2020, NLPCC.

[11]  Ying Chen,et al.  An Emotion Cause Corpus for Chinese Microblogs with Multiple-User Structures , 2017, ACM Trans. Asian Low Resour. Lang. Inf. Process..

[12]  Rui Fan,et al.  An End-to-End Multi-task Learning Network with Scope Controller for Emotion-Cause Pair Extraction , 2020, NLPCC.

[13]  Ying Chen,et al.  Hierarchical Convolution Neural Network for Emotion Cause Detection on Microblogs , 2018, ICANN.

[14]  Aytuğ Onan,et al.  Mining opinions from instructor evaluation reviews: A deep learning approach , 2019, Comput. Appl. Eng. Educ..

[15]  Akshi Kumar,et al.  Sarcasm Detection Using Soft Attention-Based Bidirectional Long Short-Term Memory Model With Convolution Network , 2019, IEEE Access.

[16]  Aytug Onan,et al.  Deep Learning Based Sentiment Analysis on Product Reviews on Twitter , 2019, Innovate-Data.

[17]  Wlodek Zadrozny,et al.  Emotion Detection in Text: a Review , 2018, ArXiv.

[18]  Diana Inkpen,et al.  Detecting Emotion Stimuli in Emotion-Bearing Sentences , 2015, CICLing.

[19]  Quoc V. Le,et al.  BAM! Born-Again Multi-Task Networks for Natural Language Understanding , 2019, ACL.

[20]  Donghong Ji,et al.  Joint multi-level attentional model for emotion detection and emotion-cause pair extraction , 2020, Neurocomputing.

[21]  Geoffrey E. Hinton,et al.  Distilling the Knowledge in a Neural Network , 2015, ArXiv.

[22]  Qin Lu,et al.  A Question Answering Approach for Emotion Cause Extraction , 2017, EMNLP.

[23]  Aytug Onan,et al.  A Term Weighted Neural Language Model and Stacked Bidirectional LSTM Based Framework for Sarcasm Identification , 2021, IEEE Access.

[24]  Ruifeng Xu,et al.  Emotion-Cause Pair Extraction as Sequence Labeling Based on a Novel Tagging Scheme , 2020, EMNLP.

[25]  Hua Xu,et al.  Emotion Cause Detection for Chinese Micro-Blogs Based on ECOCC Model , 2015, PAKDD.

[26]  Qin Lu,et al.  An ensemble approach for emotion cause detection with event extraction and multi-kernel SVMs , 2017, Tsinghua Science and Technology.

[27]  Rui Xia,et al.  RTHN: A RNN-Transformer Hierarchical Network for Emotion Cause Extraction , 2019, IJCAI.

[28]  Ying Chen,et al.  Joint Learning for Emotion Classification and Emotion Cause Detection , 2018, EMNLP.

[29]  Jianping Wang,et al.  A Unified Sequence Labeling Model for Emotion Cause Pair Extraction , 2020, COLING.

[30]  Hongliang Bi,et al.  ECSP: A New Task for Emotion-Cause Span-Pair Extraction and Classification , 2020, ArXiv.

[31]  Heyan Huang,et al.  Combining External Sentiment Knowledge for Emotion Cause Detection , 2019, NLPCC.

[32]  Jing Shan,et al.  A New Component of Interactive Multi-task Network Model for Emotion-Cause Pair Extraction , 2020, Journal of Physics: Conference Series.

[33]  Lidong Bing,et al.  A Knowledge Regularized Hierarchical Approach for Emotion Cause Analysis , 2019, EMNLP.

[34]  Muhammad Abdul-Mageed,et al.  EmoNet: Fine-Grained Emotion Detection with Gated Recurrent Neural Networks , 2017, ACL.

[35]  Yafeng Yin,et al.  A Symmetric Local Search Network for Emotion-Cause Pair Extraction , 2020, COLING.

[36]  Salwani Abdullah,et al.  Approaches to Cross-Domain Sentiment Analysis: A Systematic Literature Review , 2017, IEEE Access.

[37]  Irene Russo,et al.  EMOCause: an easy-adaptable approach to emotion cause contexts , 2011 .

[38]  Liang Yang,et al.  Extracting Emotion Causes Using Learning to Rank Methods From an Information Retrieval Perspective , 2019, IEEE Access.

[39]  Shoushan Li,et al.  End-to-End Emotion-Cause Pair Extraction with Graph Convolutional Network , 2020, COLING.

[40]  Aytuğ Onan,et al.  Two-Stage Topic Extraction Model for Bibliometric Data Analysis Based on Word Embeddings and Clustering , 2019, IEEE Access.

[41]  Jianfei Yu,et al.  End-to-End Emotion-Cause Pair Extraction Based on Sliding Window Multi-Label Learning , 2020, EMNLP.

[42]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[43]  Chu-Ren Huang,et al.  A Text-driven Rule-based System for Emotion Cause Detection , 2010, HLT-NAACL 2010.

[44]  Barbara Plank,et al.  When is multitask learning effective? Semantic sequence prediction under varying data conditions , 2016, EACL.

[45]  Wenji Mao,et al.  Context-Aware Multi-View Attention Networks for Emotion Cause Extraction , 2019, 2019 IEEE International Conference on Intelligence and Security Informatics (ISI).

[46]  Ruifeng Xu,et al.  Transition-based Directed Graph Construction for Emotion-Cause Pair Extraction , 2020, ACL.

[47]  Yu Zhou,et al.  Event-Driven Emotion Cause Extraction with Corpus Construction , 2016, EMNLP.

[48]  Xiaochao Fan,et al.  Multi-granularity bidirectional attention stream machine comprehension method for emotion cause extraction , 2019, Neural Computing and Applications.

[49]  Daling Wang,et al.  Context-aware emotion cause analysis with multi-attention-based neural network , 2019, Knowl. Based Syst..

[50]  Yu Zhou,et al.  Emotion Cause Detection with Linguistic Construction in Chinese Weibo Text , 2014, NLPCC.

[51]  Jiahao Zhao,et al.  Effective Inter-Clause Modeling for End-to-End Emotion-Cause Pair Extraction , 2020, ACL.

[52]  Sudeep TanwarSudhanshu TyagiNeeraj Kumar.  Multimedia Big Data Computing for IoT Applications , 2020 .

[53]  Saurabh Pal,et al.  Facial Emotion Detection to Assess Learner's State of Mind in an Online Learning System , 2020, ICIIT.

[54]  Rui Xia,et al.  From Independent Prediction to Re-ordered Prediction: Integrating Relative Position and Global Label Information to Emotion Cause Identification , 2019, AAAI.

[55]  Aytuğ Onan.  Sentiment Analysis in Turkish Based on Weighted Word Embeddings , 2020, 2020 28th Signal Processing and Communications Applications Conference (SIU).

[56]  Zachary Chase Lipton,et al.  Born Again Neural Networks , 2018, ICML.

[57]  Rui Xia,et al.  Emotion-Cause Pair Extraction: A New Task to Emotion Analysis in Texts , 2019, ACL.

[58]  Dawei Song,et al.  An end-to-end multi-task learning to link framework for emotion-cause pair extraction , 2020, 2021 International Conference on Image, Video Processing, and Artificial Intelligence.

[59]  Aytuğ Onan,et al.  Sentiment analysis on product reviews based on weighted word embeddings and deep neural networks , 2020, Concurr. Comput. Pract. Exp..

[60]  Jiasong Wu,et al.  Instance Segmentation Network With Self-Distillation for Scene Text Detection , 2020, IEEE Access.