Joshua 5.0: Sparser, Better, Faster, Server

We describe improvements made over the past year to Joshua, an open-source translation system for parsing-based machine translation. The main contributions this past year are significant improvements in both speed and usability of the grammar extraction and decoding steps. We have also rewritten the decoder to use a sparse feature representation, enabling training of large numbers of features with discriminative training methods.

[1]  Benjamin Van Durme.  Jerboa: A Toolkit for Randomized and Streaming Algorithms , 2012 .

[2]  Philipp Koehn,et al.  Europarl: A Parallel Corpus for Statistical Machine Translation , 2005, MTSUMMIT.

[3]  Adam Lopez,et al.  Joshua 3.0: Syntax-based Machine Translation with the Thrax Grammar Extractor , 2011, WMT@EMNLP.

[4]  Jianfeng Gao,et al.  Domain Adaptation via Pseudo In-Domain Data Selection , 2011, EMNLP.

[5]  Jean-Cédric Chappelier,et al.  A Generalized CYK Algorithm for Parsing Stochastic CFG , 1998, TAPD.

[6]  George F. Foster,et al.  Batch Tuning Strategies for Statistical Machine Translation , 2012, NAACL.

[7]  Dan Klein,et al.  Learning Accurate, Compact, and Interpretable Tree Annotation , 2006, ACL.

[8]  David Chiang,et al.  Hierarchical Phrase-Based Translation , 2007, CL.

[9]  Franz Josef Och,et al.  Minimum Error Rate Training in Statistical Machine Translation , 2003, ACL.

[10]  Mark Hopkins,et al.  SCFG Decoding Without Binarization , 2010, EMNLP.

[11]  Philipp Koehn,et al.  Scalable Modified Kneser-Ney Language Model Estimation , 2013, ACL.

[12]  Dan Klein,et al.  Faster and Smaller N-Gram Language Models , 2011, ACL.

[13]  Chris Callison-Burch,et al.  PPDB: The Paraphrase Database , 2013, NAACL.

[14]  Philip Resnik,et al.  Online Large-Margin Training of Syntactic and Structural Translation Features , 2008, EMNLP.

[15]  Chris Callison-Burch,et al.  Demonstration of Joshua: An Open Source Toolkit for Parsing-based Machine Translation , 2009, ACL.

[16]  Andreas Zollmann,et al.  Syntax Augmented Machine Translation via Chart Parsing , 2006, WMT@HLT-NAACL.

[17]  John DeNero,et al.  Asynchronous Binarization for Synchronous Grammars , 2009, ACL/IJCNLP.

[18]  Kenneth Heafield,et al.  KenLM: Faster and Smaller Language Model Queries , 2011, WMT@EMNLP.

[19]  Matt Post,et al.  Joshua 4.0: Packing, PRO, and Paraphrases , 2012, WMT@NAACL-HLT.

[20]  Ben Taskar,et al.  An End-to-End Discriminative Approach to Machine Translation , 2006, ACL.

[21]  Daniel Gildea,et al.  Synchronous Binarization for Machine Translation , 2006, NAACL.

[22]  David Chiang,et al.  A Hierarchical Phrase-Based Model for Statistical Machine Translation , 2005, ACL.

[23]  Chris Callison-Burch,et al.  Joshua 2.0: A Toolkit for Parsing-Based Machine Translation with Syntax, Semirings, Discriminative Training and Other Goodies , 2010, WMT@ACL.

[24]  Mark Hopkins,et al.  Tuning as Ranking , 2011, EMNLP.

[25]  William D. Lewis,et al.  Intelligent Selection of Language Model Training Data , 2010, ACL.

[26]  Hermann Ney,et al.  Improved Statistical Alignment Models , 2000, ACL.

[27]  Ben Taskar,et al.  Alignment by Agreement , 2006, NAACL.

[28]  Benjamin Van Durme,et al.  Annotated Gigaword , 2012, AKBC-WEKEX@NAACL-HLT.

[29]  Tong Zhang,et al.  A Discriminative Global Training Algorithm for Statistical MT , 2006, ACL.

[30]  Daniel Marcu,et al.  Scalable Inference and Training of Context-Rich Syntactic Translation Models , 2006, ACL.

[31]  Andreas Stolcke,et al.  SRILM - an extensible language modeling toolkit , 2002, INTERSPEECH.