Modeling Input Uncertainty in Neural Network Dependency Parsing

Recently introduced neural network parsers allow for new approaches to circumvent data sparsity issues by modeling character level information and by exploiting raw data in a semi-supervised setting. Data sparsity is especially prevailing when transferring to non-standard domains. In this setting, lexical normalization has often been used in the past to circumvent data sparsity. In this paper, we investigate whether these new neural approaches provide similar functionality as lexical normalization, or whether they are complementary. We provide experimental results which show that a separate normalization component improves performance of a neural network parser even if it has access to character level information as well as external word embeddings. Further improvements are obtained by a straightforward but novel approach in which the top-N best candidates provided by the normalization component are available to the parser.

[1]  Nizar Habash,et al.  CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies , 2017, CoNLL.

[2]  Timothy Baldwin,et al.  Lexical Normalisation of Short Text Messages: Makn Sens a #twitter , 2011, ACL.

[3]  Jennifer Foster.  "cba to check the spelling": Investigating Parser Performance on Discussion Forum Posts , 2010, HLT-NAACL.

[4]  Eliyahu Kiperwasser,et al.  Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations , 2016, TACL.

[5]  Markus Dickinson,et al.  Does Size Matter? Text and Grammar Revision for Parsing Social Media Data , 2013 .

[6]  Joakim Nivre,et al.  From Raw Text to Universal Dependencies - Look, No Tags! , 2017, CoNLL.

[7]  Josef van Genabith,et al.  From News to Comment: Resources and Benchmarks for Parsing the Language of Web 2.0 , 2011, IJCNLP.

[8]  Noah A. Smith,et al.  Transition-Based Dependency Parsing with Stack Long Short-Term Memory , 2015, ACL.

[9]  Tyler Baldwin,et al.  An In-depth Analysis of the Effect of Text Normalization in Social Media , 2015, HLT-NAACL.

[10]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[11]  Joakim Nivre,et al.  Arc-Hybrid Non-Projective Dependency Parsing with a Static-Dynamic Oracle , 2017, IWPT.

[12]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[13]  Yijia Liu,et al.  Parsing Tweets into Universal Dependencies , 2018, NAACL.

[14]  Joachim Daiber,et al.  The Denoised Web Treebank: Evaluating Dependency Parsing under Noisy Input Conditions , 2016, LREC.

[15]  Noah A. Smith,et al.  A Dependency Parser for Tweets , 2014, EMNLP.

[16]  Joseph Le Roux,et al.  Foreebank: Syntactic Analysis of Customer Support Forums , 2015, EMNLP.

[17]  Yang Liu,et al.  Joint POS Tagging and Text Normalization for Informal Text , 2015, IJCAI.

[18]  Brendan T. O'Connor,et al.  Twitter Universal Dependency Parsing for African-American and Mainstream American English , 2018, ACL.

[19]  Yang Liu,et al.  Improving Text Normalization via Unsupervised Model and Discriminative Reranking , 2014, ACL.

[20]  Gertjan van Noord,et al.  Parser Adaptation for Social Media by Integrating Normalization , 2017, ACL.

[21]  Brendan T. O'Connor,et al.  Improved Part-of-Speech Tagging for Online Conversational Text with Word Clusters , 2013, NAACL.

[22]  Samuel R. Bowman,et al.  A Gold Standard Dependency Corpus for English , 2014, LREC.

[23]  Gertjan van Noord,et al.  MoNoise: Modeling Noise Using a Modular Normalization System , 2017, ArXiv.

[24]  Danqi Chen,et al.  A Fast and Accurate Dependency Parser using Neural Networks , 2014, EMNLP.

[25]  Giorgio Satta,et al.  Dynamic Programming Algorithms for Transition-Based Dependency Parsers , 2011, ACL.

[26]  Dat Quoc Nguyen,et al.  A Novel Neural Network Model for Joint POS Tagging and Graph-based Dependency Parsing , 2017, CoNLL.

[27]  Noah A. Smith,et al.  Improved Transition-based Parsing by Modeling Characters instead of Words with LSTMs , 2015, EMNLP.

[28]  Slav Petrov,et al.  Overview of the 2012 Shared Task on Parsing the Web , 2012 .