SHOMA at Parseme Shared Task on Automatic Identification of VMWEs: Neural Multiword Expression Tagging with High Generalisation

This paper presents a language-independent deep learning architecture adapted to the task of multiword expression (MWE) identification. We employ a neural architecture comprising of convolutional and recurrent layers with the addition of an optional CRF layer at the top. This system participated in the open track of the Parseme shared task on automatic identification of verbal MWEs due to the use of pre-trained wikipedia word embeddings. It outperformed all participating systems in both open and closed tracks with the overall macro-average MWE-based F1 score of 58.09 averaged among all languages. A particular strength of the system is its superior performance on unseen data entries.

[1]  Virendrakumar C. Bhavsar,et al.  Deep Learning Models For Multiword Expression Identification , 2017, *SEM.

[2]  Wei Xu,et al.  Bidirectional LSTM-CRF Models for Sequence Tagging , 2015, ArXiv.

[3]  Sebastian Ruder,et al.  Fine-tuned Language Models for Text Classification , 2018, ArXiv.

[4]  Murat Saraclar,et al.  Morphology-based and sub-word language modeling for Turkish speech recognition , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[5]  Timothy Baldwin,et al.  Big Data Small Data, In Domain Out-of Domain, Known Word Unknown Word: The Impact of Word Representations on Sequence Labelling Tasks , 2015, CoNLL.

[6]  Timm Lichte,et al.  Mumpitz at PARSEME Shared Task 2018: A Bidirectional LSTM for the Identification of Verbal Multiword Expressions , 2018, LAW-MWE-CxG@COLING.

[7]  Timothy Baldwin,et al.  Multiword Expressions: A Pain in the Neck for NLP , 2002, CICLing.

[8]  Carlos Ramisch,et al.  Survey: Multiword Expression Processing: A Survey , 2017, CL.

[9]  Marie Candito,et al.  The ATILF-LLF System for Parseme Shared Task: a Transition-based Verbal Multiword Expression Tagger , 2017, MWE@EACL.

[10]  Carl Vogel,et al.  Detection of Verbal Multi-Word Expressions via Conditional Random Fields with Syntactic Dependency Features and Semantic Re-Ranking , 2017, MWE@EACL.

[11]  Tunga Güngör,et al.  Deep-BGT at PARSEME Shared Task 2018: Bidirectional LSTM-CRF Model for Verbal Multiword Expression Identification , 2018, LAW-MWE-CxG@COLING.

[12]  Jakub Waszczuk.  TRAVERSAL at PARSEME Shared Task 2018: Identification of Verbal Multiword Expressions Using a Discriminative Tree-Structured Model , 2018, LAW-MWE-CxG@COLING.

[13]  Eduard H. Hovy,et al.  End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF , 2016, ACL.

[14]  Hinrich Schütze,et al.  Word Space , 1992, NIPS.

[15]  Tiberiu Boros,et al.  GBD-NER at PARSEME Shared Task 2018: Multi-Word Expression Detection Using Bidirectional Long-Short-Term Memory Networks and Graph-Based Decoding , 2018, LAW-MWE-CxG@COLING.

[16]  Yoshua Bengio,et al.  Word Representations: A Simple and General Method for Semi-Supervised Learning , 2010, ACL.

[17]  Nathan Schneider,et al.  SemEval-2016 Task 10: Detecting Minimal Semantic Units and their Meanings (DiMSUM) , 2016, *SEMEVAL.

[18]  Jörg Tiedemann,et al.  Character-based Joint Segmentation and POS Tagging for Chinese using Bidirectional RNN-CRF , 2017, IJCNLP.

[19]  Veronika Vincze,et al.  Multiword Expressions and Named Entities in the Wiki50 Corpus , 2011, RANLP.

[20]  Tomas Mikolov,et al.  Enriching Word Vectors with Subword Information , 2016, TACL.

[21]  Guillaume Lample,et al.  Neural Architectures for Named Entity Recognition , 2016, NAACL.

[22]  Carlos Ramisch,et al.  Veyn at PARSEME Shared Task 2018: Recurrent Neural Networks for VMWE Identification , 2018, LAW-MWE-CxG@COLING.

[23]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[24]  Laura Kallmeyer,et al.  TRAPACC and TRAPACCS at PARSEME Shared Task 2018: Neural Transition Tagging of Verbal Multiword Expressions , 2018, LAW-MWE-CxG@COLING.

[25]  Noah A. Smith,et al.  Discriminative Lexical Semantic Segmentation with Gaps: Running the MWE Gamut , 2014, TACL.

[26]  Christopher D. Manning,et al.  Multiword Expression Identification with Tree Substitution Grammars: A Parsing tour de force with French , 2011, EMNLP.

[27]  Emily M. Bender,et al.  Beauty and the Beast: What Running a Broad-coverage precision grammar over the BNC taught us about the grammar and the corpus , 2005 .

[28]  Yulia Tsvetkov,et al.  Identification of Multiword Expressions by Combining Multiple Linguistic Information Sources , 2014, Computational Linguistics.

[29]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[30]  Behrang Q. Zadeh,et al.  The PARSEME Shared Task on Automatic Identification of Verbal Multiword Expressions , 2017, MWE@EACL.