SASICM A Multi-Task Benchmark For Subtext Recognition

Subtext is a kind of deep semantics which can be acquired after one or more rounds of expression transformation. As a popular way of expressing one’s intentions, it is well worth studying. In this paper, we try to make computers understand whether there is a subtext by means of machine learning. We build a Chinese dataset whose source data comes from the popular social media (e.g. Weibo, Netease Music, Zhihu, and Bilibili). In addition, we also build a baseline model called SASICM to deal with subtext recognition. The F1 score of SASICMg, whose pretrained model is GloVe, is as high as 64.37%, which is 3.97% higher than that of BERT based model, 12.7% higher than that of traditional methods on average, including support vector machine, logistic regression classifier, maximum entropy classifier, naive bayes classifier and decision tree and 2.39% higher than that of the state-of-the-art, including MARIN and BTM. The F1 score of SASICMBERT , whose pretrained model is BERT, is 65.12%, which is 0.75% higher than that of SASICMg. The accuracy rates of SASICMg and SASICMBERT are 71.16% and 70.76%, respectively, which can compete with those of other methods which are mentioned before.

[1]  Neel Kant,et al.  Practical Text Classification With Large Pre-Trained Language Models , 2018, ArXiv.

[2]  Lei Zhang,et al.  Sentiment Analysis and Opinion Mining , 2017, Encyclopedia of Machine Learning and Data Mining.

[3]  Shalom Lappin,et al.  Predicting Human Metaphor Paraphrase Judgments with Deep Neural Networks , 2018, Fig-Lang@NAACL-HLT.

[4]  Shuai Wang,et al.  Deep learning for sentiment analysis: A survey , 2018, WIREs Data Mining Knowl. Discov..

[5]  Ruifeng Xu,et al.  Context-aware Embedding for Targeted Aspect-based Sentiment Analysis , 2019, ACL.

[6]  Preslav Nakov,et al.  SemEval-2016 Task 4: Sentiment Analysis in Twitter , 2016, *SEMEVAL.

[7]  R. Plutchik.  Emotions : a general psychoevolutionary theory , 1984 .

[8]  Ivan Bondarenko,et al.  Conditional Random Fields for Metaphor Detection , 2018, Fig-Lang@NAACL-HLT.

[9]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[10]  Carolyn Penstein Rosé,et al.  Metaphor Detection with Topic Transition, Emotion and Cognition in Context , 2016, ACL.

[11]  Enkhbold Bataa,et al.  An Investigation of Transfer Learning-Based Sentiment Analysis in Japanese , 2019, ACL.

[12]  Patrik Lambert,et al.  Attention and Lexicon Regularized LSTM for Aspect-based Sentiment Analysis , 2019, ACL.

[13]  Pushpak Bhattacharyya,et al.  Automatic Sarcasm Detection: A Survey , 2016 .

[14]  Martha Palmer,et al.  Leveraging Syntactic Constructions for Metaphor Identification , 2018, Fig-Lang@NAACL-HLT.

[15]  Ning Jin,et al.  Multi-Task Learning Model Based on Multi-Scale CNN and LSTM for Sentiment Classification , 2020, IEEE Access.

[16]  Ekaterina Shutova,et al.  Grasping the Finer Point: A Supervised Similarity Network for Metaphor Detection , 2017, EMNLP.

[17]  Pushpak Bhattacharyya,et al.  Learning Cognitive Features from Gaze Data for Sentiment and Sarcasm Classification using Convolutional Neural Network , 2017, ACL.

[18]  Yue Zhang,et al.  Tweet Sarcasm Detection Using Deep Neural Network , 2016, COLING.

[19]  Md. Shad Akhtar,et al.  Multi-task learning for aspect term extraction and aspect sentiment classification , 2020, Neurocomputing.

[20]  Pushpak Bhattacharyya,et al.  Harnessing Cognitive Features for Sarcasm Detection , 2016, ACL.

[21]  Paolo Rosso,et al.  IDAT at FIRE2019: Overview of the Track on Irony Detection in Arabic Tweets , 2019, FIRE.

[22]  A. Feinstein,et al.  High agreement but low kappa: I. The problems of two paradoxes. , 1990, Journal of clinical epidemiology.

[23]  Rada Mihalcea,et al.  CASCADE: Contextual Sarcasm Detection in Online Discussion Forums , 2018, COLING.

[24]  Erik Cambria,et al.  Sentiment and Sarcasm Classification With Multitask Learning , 2019, IEEE Intelligent Systems.

[25]  Jason Baldridge,et al.  Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns , 2018, TACL.

[26]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[27]  Preslav Nakov,et al.  SemEval-2013 Task 2: Sentiment Analysis in Twitter , 2013, *SEMEVAL.

[28]  Debanjan Ghosh,et al.  A Report on the 2020 Sarcasm Detection Shared Task , 2020, FIGLANG.

[29]  Shu-Kai Hsieh,et al.  Sarcasm Detection in Chinese Using a Crowdsourced Corpus , 2016, ROCLING.

[30]  Ron Artstein.  Inter-Coder Agreement for Computational Linguistics , 2008 .

[31]  Frank Guerin,et al.  End-to-End Sequential Metaphor Identification Inspired by Linguistic Theories , 2019, ACL.

[32]  J. Sim,et al.  The kappa statistic in reliability studies: use, interpretation, and sample size requirements. , 2005, Physical therapy.

[33]  Rui Mao,et al.  Word Embedding and WordNet Based Metaphor Identification and Interpretation , 2018, ACL.

[34]  Jian Su,et al.  Reasoning with Sarcasm by Reading In-Between , 2018, ACL.

[35]  Benjamin Roth,et al.  Joint Aspect and Polarity Classification for Aspect-based Sentiment Analysis with End-to-End Neural Networks , 2018, EMNLP.

[36]  Mikhail Khodak,et al.  A Large Self-Annotated Corpus for Sarcasm , 2017, LREC.

[37]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[38]  Zhifang Sui,et al.  Towards Fine-grained Text Sentiment Transfer , 2019, ACL.

[39]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .