Deep Reinforcement Learning for Chinese Zero Pronoun Resolution

Deep neural network models for Chinese zero pronoun resolution learn semantic information for zero pronoun and candidate antecedents, but tend to be short-sighted---they often make local decisions. They typically predict coreference chains between the zero pronoun and one single candidate antecedent one link at a time, while overlooking their long-term influence on future decisions. Ideally, modeling useful information of preceding potential antecedents is critical when later predicting zero pronoun-candidate antecedent pairs. In this study, we show how to integrate local and global decision-making by exploiting deep reinforcement learning models. With the help of the reinforcement learning agent, our model learns the policy of selecting antecedents in a sequential manner, where useful information provided by earlier predicted antecedents could be utilized for making later coreference decisions. Experimental results on OntoNotes 5.0 dataset show that our technique surpasses the state-of-the-art models.

[1]  Yuji Matsumoto,et al.  Zero-anaphora resolution by learning rich syntactic pattern features , 2007, TALIP.

[2]  Tsutomu Hirao,et al.  Japanese Zero Pronoun Resolution based on Ranking Rules and Machine Learning , 2003, EMNLP.

[3]  Martha Palmer,et al.  Korean zero pronouns: analysis and resolution , 2006 .

[4]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[5]  Regina Barzilay,et al.  Language Understanding for Text-based Games using Deep Reinforcement Learning , 2015, EMNLP.

[6]  R. J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[7]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[8]  Weinan Zhang,et al.  A Deep Neural Network for Chinese Zero Pronoun Resolution , 2016, IJCAI.

[9]  Weinan Zhang,et al.  Chinese Zero Pronoun Resolution with Deep Memory Network , 2017, EMNLP.

[10]  Zhang Yu,et al.  A joint model for ellipsis identification and recovery , 2015 .

[11]  Yuji Matsumoto,et al.  Exploiting Syntactic Patterns as Clues in Zero-Anaphora Resolution , 2006, ACL.

[12]  Chen Chen,et al.  Chinese Zero Pronoun Resolution with Deep Neural Networks , 2016, ACL.

[13]  Fang Kong,et al.  A Tree Kernel-Based Unified Framework for Chinese Zero Anaphora Resolution , 2010, EMNLP.

[14]  Christopher D. Manning,et al.  Deep Reinforcement Learning for Mention-Ranking Coreference Models , 2016, EMNLP.

[15]  Sadao Kurohashi,et al.  A Discriminative Approach to Japanese Zero Anaphora Resolution with Large-scale Lexicalized Case Frames , 2011, IJCNLP.

[16]  Massimo Poesio,et al.  A Cross-Lingual ILP Solution to Zero Anaphora Resolution , 2011, ACL.

[17]  Hwee Tou Ng,et al.  Identification and Resolution of Chinese Zero Pronouns: A Machine Learning Approach , 2007, EMNLP.

[18]  Chen Chen,et al.  Chinese Zero Pronoun Resolution: An Unsupervised Approach Combining Ranking and Integer Linear Programming , 2014, AAAI.

[19]  Jianfeng Gao,et al.  Deep Reinforcement Learning for Dialogue Generation , 2016, EMNLP.

[20]  Chen Chen,et al.  Chinese Zero Pronoun Resolution: Some Recent Advances , 2013, EMNLP.

[21]  Regina Barzilay,et al.  Learning to Win by Reading Manuals in a Monte-Carlo Framework , 2011, ACL.

[22]  Chen Chen,et al.  Chinese Zero Pronoun Resolution: A Joint Unsupervised Discourse-Aware Model Rivaling State-of-the-Art Resolvers , 2015, ACL.

[23]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[24]  Jong-Hoon Oh,et al.  Intra-Sentential Subject Zero Anaphora Resolution using Multi-Column Convolutional Neural Network , 2016, EMNLP.

[25]  Iryna Gurevych,et al.  Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging , 2017, EMNLP.

[26]  Jong-Hoon Oh,et al.  Intra-sentential Zero Anaphora Resolution using Subject Sharing Recognition , 2015, EMNLP.

[27]  Ting Liu,et al.  Generating and Exploiting Large-scale Pseudo Training Data for Zero Pronoun Resolution , 2016, ACL.

[28]  Wenhan Xiong,et al.  DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning , 2017, EMNLP.

[29]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[30]  Regina Barzilay,et al.  Improving Information Extraction by Acquiring External Evidence with Reinforcement Learning , 2016, EMNLP.