Intra-Sentential Subject Zero Anaphora Resolution using Multi-Column Convolutional Neural Network

This paper proposes a method for intrasentential subject zero anaphora resolution in Japanese. Our proposed method utilizes a Multi-column Convolutional Neural Network (MCNN) for predicting zero anaphoric relations. Motivated by Centering Theory and other previous works, we exploit as clues both the surface word sequence and the dependency tree of a target sentence in our MCNN. Even though the F-score of our method was lower than that of the state-of-the-art method, which achieved relatively high recall and low precision, our method achieved much higher precision (>0.8) in a wide range of recall levels. We believe such high precision is crucial for real-world NLP applications and thus our method is preferable to the state-of-the-art method.

[1]  R. Iida,et al.  Incorporating Contextual Cues in Trainable Models for Coreference Resolution , 2003 .

[2]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[3]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[4]  Yuji Matsumoto,et al.  Exploiting Syntactic Patterns as Clues in Zero-Anaphora Resolution , 2006, ACL.

[5]  Wanxiang Che,et al.  Convolution Neural Network for Relation Extraction , 2013, ADMA.

[6]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[7]  Sadao Kurohashi,et al.  A Discriminative Approach to Japanese Zero Anaphora Resolution with Large-scale Lexicalized Case Frames , 2011, IJCNLP.

[8]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[9]  Bowen Zhou,et al.  Classifying Relations by Ranking with Convolutional Neural Networks , 2015, ACL.

[10]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[11]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Jun Zhao,et al.  Relation Classification via Convolutional Deep Neural Network , 2014, COLING.

[13]  Shigeko Nariyama,et al.  Grammar for ellipsis resolution in Japanese. , 2002, TMI.

[14]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[15]  Hang Li,et al.  Convolutional Neural Network Architectures for Matching Natural Language Sentences , 2014, NIPS.

[16]  Vincent Ng,et al.  Learning Noun Phrase Anaphoricity to Improve Conference Resolution: Issues in Representation and Optimization , 2004, ACL.

[17]  Razvan Pascanu,et al.  Theano: new features and speed improvements , 2012, ArXiv.

[18]  Jong-Hoon Oh,et al.  Intra-sentential Zero Anaphora Resolution using Subject Sharing Recognition , 2015, EMNLP.

[19]  Sven Behnke,et al.  Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition , 2010, ICANN.

[20]  Massimo Poesio,et al.  A Cross-Lingual ILP Solution to Zero Anaphora Resolution , 2011, ACL.

[21]  Yuji Matsumoto,et al.  Applying Conditional Random Fields to Japanese Morphological Analysis , 2004, EMNLP.

[22]  Yuji Matsumoto,et al.  Annotating a Japanese Text Corpus with Predicate-Argument and Coreference Relations , 2007, LAW@ACL.

[23]  Wenpeng Yin,et al.  Convolutional Neural Network for Paraphrase Identification , 2015, NAACL.

[24]  Masaaki Nagata,et al.  A Japanese Predicate Argument Structure Analysis using Decision Lists , 2008, EMNLP.

[25]  Ming Zhou,et al.  Question Answering over Freebase with Multi-Column Convolutional Neural Networks , 2015, ACL.

[26]  Tsutomu Hirao,et al.  Japanese Zero Pronoun Resolution based on Ranking Rules and Machine Learning , 2003, EMNLP.

[27]  Ralph Grishman,et al.  Relation Extraction: Perspective from Convolutional Neural Networks , 2015, VS@HLT-NAACL.

[28]  Tong Zhang,et al.  Effective Use of Word Order for Text Categorization with Convolutional Neural Networks , 2014, NAACL.

[29]  Hiroyuki Shindo,et al.  Joint Case Argument Identification for Japanese Predicate Argument Structure Analysis , 2015, ACL.

[30]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[31]  Yuji Matsumoto,et al.  Japanese Predicate Argument Structure Analysis Exploiting Argument Position and Type , 2011, IJCNLP.

[32]  Yuji Matsumoto,et al.  Jointly Extracting Japanese Predicate-Argument Relation with Markov Logic , 2011, IJCNLP.

[33]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[34]  Jun Zhao,et al.  Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks , 2015, EMNLP.

[35]  Kazuhiro Seki,et al.  A Probabilistic Method for Analyzing Japanese Anaphora Integrating Zero Pronoun Detection and Resolution , 2002, COLING.

[36]  Scott Weinstein,et al.  Centering: A Framework for Modeling the Local Coherence of Discourse , 1995, CL.

[37]  Masaru Kitsuregawa,et al.  Polynomial to Linear: Efficient Classification with Conjunctive Features , 2009, EMNLP.

[38]  Manabu Okumura,et al.  Zero Pronoun Resolution in Japanese Discourse Based on Centering Theory , 1996, COLING.

[39]  Daisuke Kawahara,et al.  A Fully-Lexicalized Probabilistic Model for Japanese Zero Anaphora Resolution , 2008, COLING.

[40]  Jason Weston,et al.  Learning Anaphoricity and Antecedent Ranking Features for Coreference Resolution , 2015, ACL.

[41]  Tomoko Izumi,et al.  Discriminative Approach to Predicate-Argument Structure Analysis with Zero-Anaphora Resolution , 2009, ACL.