WiC = TSV = WSD: On the Equivalence of Three Semantic Tasks

The Word-in-Context (WiC) task has attracted considerable attention in the NLP community, as demonstrated by the popularity of the recent MCL-WiC SemEval shared task. Systems and lexical resources from word sense disambiguation (WSD) are often used for the WiC task and WiC dataset construction. In this paper, we establish the exact relationship between WiC and WSD, as well as the related task of target sense verification (TSV). Building upon a novel hypothesis on the equivalence of sense and meaning distinctions, we demonstrate through the applications of tools from theoretical computer science that these three semantic classification problems can be pairwise reduced to each other, and therefore are equivalent. The results of experiments that involve systems and datasets for both WiC and WSD provide strong empirical evidence that our problem reductions work in practice.

[1]  Simone Paolo Ponzetto,et al.  BabelNet: Building a Very Large Multilingual Semantic Network , 2010, ACL.

[2]  Artem Revenko,et al.  WiC-TSV: An Evaluation Benchmark for Target Sense Verification of Words in Context , 2020, ArXiv.

[3]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[4]  Ashutosh Modi,et al.  MCL@IITK at SemEval-2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation using Augmented Data, Signals, and Transformers , 2021, SEMEVAL.

[5]  Simone Paolo Ponzetto,et al.  BabelNet: The automatic construction, evaluation and application of a wide-coverage multilingual semantic network , 2012, Artif. Intell..

[6]  Alessandro Raganato,et al.  XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization , 2020, EMNLP.

[7]  José Camacho-Collados,et al.  WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations , 2018, NAACL.

[8]  Omer Levy,et al.  SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems , 2019, NeurIPS.

[9]  Roberto Navigli,et al.  Word sense disambiguation: A survey , 2009, CSUR.

[10]  Daniel Loureiro,et al.  LIAAD at SemDeep-5 Challenge: Word-in-Context (WiC) , 2019, SemDeep@IJCAI.

[11]  Tommaso Pasini,et al.  Exemplification Modeling: Can You Give Me an Example, Please? , 2021, IJCAI.

[12]  Martha Palmer,et al.  Verbnet: a broad-coverage, comprehensive verb lexicon , 2005 .

[13]  Nikolay Arefyev,et al.  LIORI at SemEval-2021 Task 2: Span Prediction and Binary Classification approaches to Word-in-Context Disambiguation , 2021, SEMEVAL.

[14]  Roberto Navigli,et al.  Meaningful Clustering of Senses Helps Boost Word Sense Disambiguation Performance , 2006, ACL.

[15]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[16]  Federico Martelli,et al.  SemEval-2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation (MCL-WiC) , 2021, SEMEVAL.

[17]  George A. Miller,et al.  A Semantic Concordance , 1993, HLT.

[18]  Tommaso Pasini,et al.  ESC: Redesigning WSD with Extractive Sense Comprehension , 2021, NAACL.

[19]  Roberto Navigli,et al.  Word Sense Disambiguation: A Unified Evaluation Framework and Empirical Comparison , 2017, EACL.

[20]  Yoav Shoham,et al.  SenseBERT: Driving Some Sense into BERT , 2019, ACL.

[21]  Helmut Schmid,et al.  Improvements in Part-of-Speech Tagging with an Application to German , 1999 .