Semi-supervised dependency parsing using generalized tri-training

Martins et al. (2008) presented what to the best of our knowledge still ranks as the best overall result on the CONLL-X Shared Task datasets. The paper shows how triads of stacked dependency parsers described in Martins et al. (2008) can label unlabeled data for each other in a way similar to co-training and produce end parsers that are significantly better than any of the stacked input parsers. We evaluate our system on five datasets from the CONLL-X Shared Task and obtain 10--20% error reductions, incl. the best reported results on four of them. We compare our approach to other semi-supervised learning algorithms.

[1]  Fabio Roli,et al.  Using Co-training and Self-training in Semi-supervised Multiple Classifier Systems , 2006, SSPR/SPR.

[2]  S. Sathiya Keerthi,et al.  Large scale semi-supervised linear SVMs , 2006, SIGIR.

[3]  Eric P. Xing,et al.  Stacking Dependency Parsers , 2008, EMNLP.

[4]  Alon Lavie,et al.  Parser Combination by Reparsing , 2006, NAACL.

[5]  Philipp Koehn,et al.  Europarl: A Parallel Corpus for Statistical Machine Translation , 2005, MTSUMMIT.

[6]  Fernando Pereira,et al.  Non-Projective Dependency Parsing using Spanning Tree Algorithms , 2005, HLT.

[7]  Mihai Surdeanu,et al.  Ensemble Models for Dependency Parsing: Cheap and Good? , 2010, HLT-NAACL.

[8]  Zhi-Hua Zhou,et al.  Improve Computer-Aided Diagnosis With Machine Learning Techniques Using Undiagnosed Samples , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[9]  Andrew S. Gordon,et al.  Clustering Words by Syntactic Similarity improves Dependency Parsing of Predicate-argument Structures , 2009, IWPT.

[10]  Lluís Màrquez i Villodre,et al.  SVMTool: A general POS Tagger Generator Based on Support Vector Machines , 2004, LREC.

[11]  Joakim Nivre,et al.  Single Malt or Blended? A Study in Multilingual Parser Optimization , 2007, EMNLP.

[12]  Sabine Buchholz,et al.  CoNLL-X Shared Task on Multilingual Dependency Parsing , 2006, CoNLL.

[13]  Dale Schuurmans,et al.  Semi-Supervised Convex Training for Dependency Parsing , 2008, ACL.

[14]  Zhi-Hua Zhou,et al.  When semi-supervised learning meets ensemble learning , 2009, MCS.

[15]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[16]  Lidan Zhang,et al.  Dependency Parsing with Energy-based Reinforcement Learning , 2009, IWPT.

[17]  Daniel Zeman,et al.  Improving Parsing Accuracy by Combining Diverse Dependency Parsers , 2005, IWPT.

[18]  Joakim Nivre,et al.  MaltParser: A Language-Independent System for Data-Driven Dependency Parsing , 2007, Natural Language Engineering.

[19]  Steven Abney,et al.  Semisupervised Learning for Computational Linguistics , 2007 .

[20]  Joakim Nivre,et al.  Integrating Graph-Based and Transition-Based Dependency Parsers , 2008, ACL.

[21]  Eugene Charniak,et al.  Effective Self-Training for Parsing , 2006, NAACL.

[22]  Hitoshi Isahara,et al.  Chinese Chunking with Tri-training Learning , 2006, ICCPOL.

[23]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[24]  Friedhelm Schwenker,et al.  Co-Training by Committee: A Generalized Framework for Semi-Supervised Learning with Committees , 2008, Int. J. Softw. Informatics.

[25]  Ruy Luiz Milidiú,et al.  Improving BAS committee performance with a semi-supervised approach , 2009, ESANN.

[26]  Anders Søgaard,et al.  Simple Semi-Supervised Training of Part-Of-Speech Taggers , 2010, ACL.

[27]  Minh Le Nguyen,et al.  Using Semi-supervised Learning for Question Classification , 2006, ICCPOL.

[28]  Joakim Nivre,et al.  Voting and Stacking in Data-Driven Dependency Parsing , 2009, NODALIDA.

[29]  Jonas Kuhn,et al.  Data-Driven Dependency Parsing of New Languages Using Incomplete and Noisy Training Data , 2009, CoNLL.

[30]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[31]  Xavier Carreras,et al.  Simple Semi-supervised Dependency Parsing , 2008, ACL.

[32]  Zhi-Hua Zhou,et al.  Tri-training: exploiting unlabeled data using three classifiers , 2005, IEEE Transactions on Knowledge and Data Engineering.

[33]  Ayhan Demiriz,et al.  Exploiting unlabeled data in ensemble methods , 2002, KDD.

[34]  Jun'ichi Tsujii,et al.  Dependency Parsing and Domain Adaptation with LR Models and Parser Ensembles , 2007, EMNLP.