Transition-Based Deep Input Linearization

Traditional methods for deep NLG adopt pipeline approaches comprising stages such as constructing syntactic input, predicting function words, linearizing the syntactic input and generating the surface forms. Though easier to visualize, pipeline approaches suffer from error propagation. In addition, information available across modules cannot be leveraged by all modules. We construct a transition-based model to jointly perform linearization, function word prediction and morphological generation, which considerably improves upon the accuracy compared to a pipelined baseline system. On a standard deep input linearization shared task, our system achieves the best results reported so far.

[1]  Michael Collins,et al.  Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms , 2002, EMNLP.

[2]  Daniel Gildea,et al.  The Proposition Bank: An Annotated Corpus of Semantic Roles , 2005, CL.

[3]  Michael White,et al.  Hypertagging: Supertagging for Surface Realization with CCG , 2008, ACL.

[4]  Kevin Knight,et al.  Generation that Exploits Corpus-Based Statistical Knowledge , 1998, ACL.

[5]  Yue Zhang.  Partial-Tree Linearization: Generalized Word Ordering for Text Synthesis , 2013, IJCAI.

[6]  Yue Zhang,et al.  A Transition-based Model for Joint Segmentation, POS-tagging and Normalization , 2015, EMNLP.

[7]  Igor Mel’čuk,et al.  Dependency Syntax: Theory and Practice , 1987 .

[8]  Igor Mel’čuk,et al.  Semantics: From meaning to text. Volume 2 , 2013 .

[9]  Alexander M. Rush,et al.  Word Ordering Without Syntax , 2016, EMNLP.

[10]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[11]  Yue Zhang,et al.  An Empirical Comparison Between N-gram and Syntactic Language Models for Word Ordering , 2015, EMNLP.

[12]  Leo Wanner,et al.  Broad Coverage Multilingual Deep Sentence Generation with a Stochastic Multi-Level Realizer , 2010, COLING.

[13]  Benoît Favre,et al.  StuMaBa : From Deep Representation to Surface , 2011, ENLG.

[14]  Joakim Nivre,et al.  Algorithms for Deterministic Incremental Dependency Parsing , 2008, CL.

[15]  Stephen Clark,et al.  Syntactic Processing Using the Generalized Perceptron and Beam Search , 2011, CL.

[16]  Yue Zhang,et al.  Joint Morphological Generation and Syntactic Linearization , 2014, AAAI.

[17]  Marcus Tomalin,et al.  Word Ordering with Phrase-Based Grammars , 2014, EACL.

[18]  Stephen Clark,et al.  Discriminative Syntax-Based Word Ordering for Text Generation , 2015, CL.

[19]  John Carroll,et al.  An Efficient Chart Generator for (Semi-)Lexicalist Grammars , 2001 .

[20]  Adam Kilgarriff,et al.  of the European Chapter of the Association for Computational Linguistics , 2006 .

[21]  Danqi Chen,et al.  A Fast and Accurate Dependency Parser using Neural Networks , 2014, EMNLP.

[22]  Joakim Nivre,et al.  Deterministic Dependency Parsing of English Text , 2004, COLING.

[23]  Heng Ji,et al.  Incremental Joint Extraction of Entity Mentions and Relations , 2014, ACL.

[24]  Michael White,et al.  Efficient Realization of Coordinate Structures in Combinatory Categorial Grammar , 2006 .

[25]  Andrew McCallum,et al.  Dynamic conditional random fields: factorized probabilistic models for labeling and segmenting sequence data , 2004, J. Mach. Learn. Res..

[26]  Ralph Grishman,et al.  Annotating Noun Argument Structure for NomBank , 2004, LREC.

[27]  Anja Belz,et al.  The First Surface Realisation Shared Task: Overview and Evaluation Results , 2011, ENLG.

[28]  Yue Zhang,et al.  Syntactic SMT Using a Discriminative Text Generation Model , 2014, EMNLP.

[29]  Stephen Clark,et al.  A Fast Decoder for Joint Word Segmentation and POS-Tagging Using a Single Discriminative Model , 2010, EMNLP.

[30]  Yue Zhang,et al.  Transition-Based Syntactic Linearization , 2015, NAACL.

[31]  Robert Dale,et al.  Building applied natural language generation systems , 1997, Natural Language Engineering.

[32]  Stephen Wan,et al.  Improving Grammaticality in Statistical Sentence Generation: Introducing a Dependency Spanning Tree Algorithm with an Argument Satisfaction Model , 2009, EACL.

[33]  Michael White,et al.  Reining in CCG Chart Realization , 2004, INLG.

[34]  Christopher D. Manning,et al.  Joint Parsing and Named Entity Recognition , 2009, NAACL.

[35]  Stephanie Seneff,et al.  Automatic grammar correction for second-language learners , 2006, INTERSPEECH.

[36]  Kristina Toutanova,et al.  A Discriminative Syntactic Word Order Model for Machine Translation , 2007, ACL.

[37]  Yue Zhang,et al.  Transition-Based Syntactic Linearization with Lookahead Features , 2016, HLT-NAACL.

[38]  Brian Roark,et al.  Incremental Parsing with the Perceptron Algorithm , 2004, ACL.

[39]  Stephan Oepen,et al.  High Efficiency Realization for a Wide-Coverage Unification Grammar , 2005, IJCNLP.

[40]  Michael White,et al.  Perceptron Reranking for CCG Realization , 2009, EMNLP.

[41]  Regina Barzilay,et al.  Sentence Fusion for Multidocument News Summarization , 2005, CL.