Filtering Reordering Table Using a Novel Recursive Autoencoder Model for Statistical Machine Translation

In phrase-based machine translation (PBMT) systems, the reordering table and phrase table are very large and redundant. Unlike most previous works which aim to filter phrase table, this paper proposes a novel deep neural network model to prune reordering table. We cast the task as a deep learning problem where we jointly train two models: a generative model to implement rule embedding and a discriminative model to classify rules. The main contribution of this paper is that we optimize the reordering model in PBMT by filtering reordering table using a recursive autoencoder model. To evaluate the performance of the proposed model, we performed it on public corpus to measure its reordering ability. The experimental results show that our approach obtains high improvement in BLEU score with less scale of reordering table on two language pairs: English-Chinese (

[1]  Bor-Sen Chen,et al.  Robust H∞ filtering for nonlinear stochastic systems , 2005 .

[2]  Jeffrey Pennington,et al.  Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection , 2011, NIPS.

[3]  Yan Lin,et al.  A unified design for state and output feedback H∞ control of nonlinear stochastic Markovian jump systems with state and disturbance-dependent noise , 2009, Autom..

[4]  Yoshua Bengio,et al.  Neural Probabilistic Language Models , 2006 .

[5]  Bor-Sen Chen,et al.  LaSalle-Type Theorem and Its Applications to Infinite Horizon Optimal Control of Discrete-Time Nonlinear Stochastic Systems , 2017, IEEE Transactions on Automatic Control.

[6]  Weihai Zhang,et al.  Nonlinear stochastic passivity, feedback equivalence and global stabilization , 2012 .

[7]  Anoop Sarkar,et al.  Lexicalized Reordering for Left-to-Right Hierarchical Phrase-based Translation , 2017, EACL.

[8]  Peng Xu,et al.  A Systematic Comparison of Phrase Table Pruning Techniques , 2012, EMNLP.

[9]  Geoffrey E. Hinton,et al.  Acoustic Modeling Using Deep Belief Networks , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[10]  Jeffrey Pennington,et al.  Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions , 2011, EMNLP.

[11]  Weihai Zhang,et al.  Stabilization of interconnected nonlinear stochastic Markovian jump systems via dissipativity approach , 2011, Autom..

[12]  Yue Yin,et al.  Phrase table filtration based on virtual context in phrase-based statistical machine translation , 2013 .

[13]  Qun Liu,et al.  Maximum Entropy Based Phrase Reordering Model for Statistical Machine Translation , 2006, ACL.

[14]  Chao Wang,et al.  Chinese Syntactic Reordering for Statistical Machine Translation , 2007, EMNLP.

[15]  Philipp Koehn,et al.  Moses: Open Source Toolkit for Statistical Machine Translation , 2007, ACL.

[16]  Andreas Stolcke,et al.  SRILM at Sixteen: Update and Outlook , 2011 .

[17]  Yang Liu,et al.  Recursive Autoencoders for ITG-Based Translation , 2013, EMNLP.

[18]  Sankar K. Pal,et al.  Multilayer perceptron, fuzzy sets, and classification , 1992, IEEE Trans. Neural Networks.

[19]  Bor-Sen Chen,et al.  On stabilizability and exact observability of stochastic systems with their applications , 2023, Autom..

[20]  Robert L. Mercer,et al.  The Mathematics of Statistical Machine Translation: Parameter Estimation , 1993, CL.

[21]  Y. Huang,et al.  Stochastic H2/H8 control for discrete-time systems with state and disturbance dependent noise. , 2007 .

[22]  Ming Zhou,et al.  Bilingually-constrained Phrase Embeddings for Machine Translation , 2014, ACL.

[23]  Weihai Zhang,et al.  Suboptimal stochastic H-two/H-infinity design with spectrum constraint , 2008 .

[24]  Leon O. Chua,et al.  The CNN paradigm , 1993 .

[25]  Daniel Marcu,et al.  Statistical Phrase-Based Translation , 2003, NAACL.

[26]  Tong Zhang,et al.  A Localized Prediction Model for Statistical Machine Translation , 2005, ACL.

[27]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.