MelBERT: Metaphor Detection via Contextualized Late Interaction using Metaphorical Identification Theories

Automated metaphor detection is a challenging task to identify the metaphorical expression of words in a sentence. To tackle this problem, we adopt pre-trained contextualized models, e.g., BERT and RoBERTa. To this end, we propose a novel metaphor detection model, namely metaphor-aware late interaction over BERT (MelBERT). Our model not only leverages contextualized word representation but also benefits from linguistic metaphor identification theories to detect whether the target word is metaphorical. Our empirical results demonstrate that MelBERT outperforms several strong baselines on four benchmark datasets, i.e., VUA-18, VUA-20, MOH-X, and TroFi.

[1]  Beata Beigman Klebanov,et al.  Go Figure! Multi-task transformer-based architecture for metaphor detection using idioms: ETS team in 2020 metaphor shared task , 2020, FIGLANG.

[2]  M. Zaharia,et al.  ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT , 2020, SIGIR.

[3]  Frank Guerin,et al.  End-to-End Sequential Metaphor Identification Inspired by Linguistic Theories , 2019, ACL.

[4]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[5]  Ruofei Zhang,et al.  TwinBERT: Distilling Knowledge to Twin-Structured Compressed BERT Models for Large-Scale Retrieval , 2020, CIKM.

[6]  Saif Mohammad,et al.  Metaphor as a Medium for Emotion: An Empirical Study , 2016, *SEMEVAL.

[7]  R. Gibbs,et al.  MIP: A method for identifying metaphorically used words in discourse , 2007 .

[8]  Philipp Koehn,et al.  Synthesis Lectures on Human Language Technologies , 2016 .

[9]  Beata Beigman Klebanov,et al.  A Report on the 2020 VUA and TOEFL Metaphor Detection Shared Task , 2020, FIGLANG.

[10]  Anoop Sarkar,et al.  A Clustering Approach for Nearly Unsupervised Recognition of Nonliteral Language , 2006, EACL.

[11]  Johannes Bjerva,et al.  Detecting novel metaphor using selectional preference information , 2016 .

[12]  Pawel Dybala,et al.  Humor, Emotions and Communication: Human-like Issues of Human-Computer Interactions , 2012, CogSci.

[13]  Fumiyo Fukumoto,et al.  DeepMet: A Reading Comprehension Paradigm for Token-level Metaphor Detection , 2020, FIGLANG.

[14]  Stephen Clark,et al.  Modelling metaphor with attribute-based semantics , 2017, EACL.

[15]  Jimmy J. Lin,et al.  Document Expansion by Query Prediction , 2019, ArXiv.

[16]  Jean Maillard,et al.  Black Holes and White Rabbits: Metaphor Identification with Visual Features , 2016, NAACL.

[17]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[18]  Tomek Strzalkowski,et al.  Using Imageability and Topic Chaining to Locate Metaphors in Linguistic Corpora , 2013, SBP.

[19]  Jamie Callan,et al.  Context-Aware Term Weighting For First Stage Passage Retrieval , 2020, SIGIR.

[20]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[21]  Chuhan Wu,et al.  Neural Metaphor Detecting with CNN-LSTM Model , 2018, Fig-Lang@NAACL-HLT.

[22]  Bhaskar Mitra,et al.  An Introduction to Neural Information Retrieval , 2018, Found. Trends Inf. Retr..

[23]  Yorick Wilks,et al.  A Preferential, Pattern-Seeking, Semantics for Natural Language Inference , 1975, Artif. Intell..

[24]  G. Lakoff,et al.  Metaphors We Live by , 1982 .

[25]  Eunsol Choi,et al.  Neural Metaphor Detection in Context , 2018, EMNLP.

[26]  Suma Bhat,et al.  IlliniMet: Illinois System for Metaphor Detection with Contextual and Linguistic Information , 2020, FIGLANG.

[27]  Gerard J. Steen,et al.  A method for linguistic metaphor identification : from MIP to MIPVU , 2010 .

[28]  Yulia Tsvetkov,et al.  Metaphor Detection with Cross-Lingual Model Transfer , 2014, ACL.

[29]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[30]  Iryna Gurevych,et al.  Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks , 2019, EMNLP.

[31]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[32]  Lin Sun,et al.  Unsupervised Metaphor Identification Using Hierarchical Graph Factorization Clustering , 2013, NAACL.

[33]  Omer Levy,et al.  GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding , 2018, BlackboxNLP@EMNLP.

[34]  Beata Beigman Klebanov,et al.  A Report on the 2018 VUA Metaphor Detection Shared Task , 2018, Fig-Lang@NAACL-HLT.

[35]  Yorick Wilks,et al.  Making Preferences More Active , 1978, Artif. Intell..

[36]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[37]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[38]  Yair Neuman,et al.  Literal and Metaphorical Sense Identification through Concrete and Abstract Context , 2011, EMNLP.

[39]  Anna Korhonen,et al.  Metaphor Identification Using Verb and Noun Clustering , 2010, COLING.

[40]  Carlo Strapparava,et al.  Metaphor: A Computational Perspective by Tony Veale, Ekaterina Shutova and Beata Beigman Klebanov , 2016, CL.

[41]  Ekaterina Shutova,et al.  Grasping the Finer Point: A Supervised Similarity Network for Metaphor Detection , 2017, EMNLP.

[42]  L. Lagerwerf,et al.  Openness in Metaphorical and Straightforward Advertisements: Appreciation Effects , 2008 .

[43]  Mike Thelwall,et al.  Sentiment Analysis Is a Big Suitcase , 2017, IEEE Intelligent Systems.

[44]  Toru Ishida,et al.  Translation Agent: A New Metaphor for Machine Translation , 2014, New Generation Computing.

[45]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.