Grammatical Profiling for Semantic Change Detection

Semantics, morphology and syntax are strongly interdependent. However, the majority of computational methods for semantic change detection use distributional word representations which encode mostly semantics. We investigate an alternative method, grammatical profiling, based entirely on changes in the morphosyntactic behaviour of words. We demonstrate that it can be used for semantic change detection and even outperforms some distributional semantic methods. We present an in-depth qualitative and quantitative analysis of the predictions made by our grammatical profiling system, showing that they are plausible and interpretable.

[1]  Milan Straka,et al.  Tokenizing, POS Tagging, Lemmatizing and Parsing UD 2.0 with UDPipe , 2017, CoNLL.

[2]  Stefan Th. Gries,et al.  Ways of trying in Russian: clustering behavioral profiles , 2006, Corpus Linguistics and Linguistic Theory.

[3]  Barbara McGillivray,et al.  SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection , 2020, SEMEVAL.

[4]  Julia Kuznetsova Linguistic Profiles: Going from Form to Meaning Via Statistics , 2015 .

[5]  Andrey Kutuzov,et al.  RuShiftEval: a shared task on semantic shift detection for Russian , 2021 .

[6]  Erik Velldal,et al.  Diachronic word embeddings and semantic shifts: a survey , 2018, COLING.

[7]  Patrick Hanks,et al.  Contextual dependency and lexical sets , 1996 .

[8]  Ignacio Iacobacci,et al.  Embeddings for Word Sense Disambiguation: An Evaluation Study , 2016, ACL.

[9]  Detection of Semantic Changes in Russian Nouns with Distributional Models and Grammatical Features , 2021 .

[10]  Mario Giulianelli,et al.  Analysing Lexical Semantic Change with Contextualised Word Representations , 2020, ACL.

[11]  Lidia Pivovarova,et al.  Scalable and Interpretable Semantic Change Detection , 2021, NAACL.

[12]  José Camacho-Collados,et al.  WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations , 2018, NAACL.

[13]  Dominik Schlechtweg,et al.  Diachronic Usage Relatedness (DURel): A Framework for the Annotation of Lexical Semantic Change , 2018, NAACL.

[14]  Tommaso Caselli,et al.  DIACR-Ita @ EVALITA2020: Overview of the EVALITA2020 Diachronic Lexical Semantics (DIACR-Ita) Task , 2020, EVALITA.

[15]  Annalina Caputo,et al.  Kronos-it: a Dataset for the Italian Semantic Change Detection Task , 2019, CLiC-it.

[16]  Olga Lyashevskaya,et al.  Grammatical profiles and the interaction of the lexicon with aspect, tense, and mood in Russian , 2011 .

[17]  Attapol Khamkhien,et al.  Lexical Priming: A New Theory of Words and Language , 2013 .

[18]  Stefan Th. Gries,et al.  Behavioral profiles : A corpus-based perspective on synonymy and antonymy , 2010 .

[19]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[20]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[21]  S. Gries,et al.  Behavioral profiles: A corpus-based approach to cognitive semantic analysis , 2009 .

[22]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[23]  B. Joseph,et al.  Language History, Language Change, and Language Relationship: An Introduction to Historical and Comparative Linguistics , 1996 .

[24]  Lidia Pivovarova,et al.  Three-part diachronic semantic change dataset for Russian , 2021, LCHANGE.

[25]  Annalina Caputo,et al.  Diachronic Analysis of Entities by Exploiting Wikipedia Page revisions , 2019, RANLP.

[26]  Marco Baroni,et al.  A distributional similarity approach to the detection of semantic change in the Google Books Ngram corpus. , 2011, GEMS.

[27]  Hanne Martine Eckhoff,et al.  Grammatical Profiles and Aspect in Old Church Slavonic , 2014 .

[28]  N. Vayatis,et al.  Selective review of offline change point detection methods , 2019 .

[29]  Patrick Juola,et al.  The Time Course of Language Change , 2003, Comput. Humanit..

[30]  Stefan Th. Gries,et al.  Assessing frequency changes in multistage diachronic corpora: Applications for historical corpus linguistics and the study of language acquisition , 2009, Lit. Linguistic Comput..

[31]  Florian Schmidt,et al.  How does BERT capture semantics? A closer look at polysemous words , 2020, BLACKBOXNLP.