Overview of the Recognizing Inference in Text (RITE-2) at NTCIR-10

This paper describes an overview of RITE-2 (Recognizing Inference in TExt) task in NTCIR-10. We evaluated systems that automatically recognize semantic relations between sentences such as paraphrase, entailment, contradiction in Japanese, Simplified Chinese and Traditional Chinese. The tasks in RITE-2 are Binary Classification of entailment (BC Subtask), Multi-Class Classification including paraphrase and contradiction (MC Subtask), Entrance Exam Subtasks (Exam BC and Exam Search), Unit Test, and RITE4QA Subtask. We had 28 active participants, and received 215 formal runs (110 Japanese runs, 53 Traditional Chinese runs, 52 Simplified Chinese runs). This paper also describes how the datasets for RITE-2 had been developed, how the systems were evaluated, and reports RITE-2 formal run results.

[1]  Sanda M. Harabagiu,et al.  Methods for Using Textual Entailment in Open-Domain Question Answering , 2006, ACL.

[2]  Wen-Lian Hsu,et al.  IASL RITE System at NTCIR-10 , 2013, NTCIR.

[3]  Takashi Ninomiya,et al.  EHIME Textual Entailment System Using Markov Logic in NTCIR-10 RITE-2 , 2013, NTCIR.

[4]  Tomoya Iwakura,et al.  FLL: Local Alignments based Approach for NTCIR-10 RITE-2 , 2013, NTCIR.

[5]  Yu-An Lin,et al.  IMTKU Textual Entailment System for Recognizing Inference in Text at NTCIR-10 RITE2 , 2011, NTCIR.

[6]  Junta Mizuno,et al.  TKDDI group at NTCIR10-RITE2: Recognizing Texual Entailment Based on Dependency Structure Alignment , 2013, NTCIR.

[7]  Doina Tatar,et al.  Summarization by Logic Segmentation and Text Entailment , 2008 .

[8]  Han Ren,et al.  The WHUTE System in NTCIR-10 RITE Task , 2011, NTCIR.

[9]  Sadao Kurohashi,et al.  Predicate-argument Structure based Textual Entailment Recognition System of KYOTO Team for NTCIR9 RITE , 2011, NTCIR.

[10]  Hayato Yamana,et al.  WSD Team's Approaches for Textual Entailment Recognition at the NTCIR10 (RITE2) , 2013, NTCIR.

[11]  Tao-Hsing Chang,et al.  KC99: A Prediction System for Chinese Textual Entailment Relation Using Decision Tree , 2013, NTCIR.

[12]  Yotaro Watanabe,et al.  THK's Natural Logic-based Compositional Textual Entailment Model at NTCIR-10 RITE-2 , 2013, NTCIR.

[13]  Han Ren,et al.  The WHUTE System in NTCIR-11 RITE-VAL Task , 2014, NTCIR.

[14]  Akira Shimazu,et al.  JAIST Participation at NTCIR-10 RITE-2 , 2013, NTCIR.

[15]  Koichi Takeuchi,et al.  Construction of a Simple Inference System of Textural Similarity (oka1 RITE2) , 2013, NTCIR.

[16]  Takashi Onishi,et al.  Detecting Contradiction in Text by Using Lexical Mismatch and Structural Similarity , 2013, NTCIR.

[17]  Chuan-Jie Lin,et al.  The Description of the NTOU RITE System in NTCIR-9 , 2011, NTCIR.

[18]  Chao-Lin Liu,et al.  NCCU-MIG at NTCIR-10: Using Lexical, Syntactic, and Semantic Features for the RITE Tasks , 2013, NTCIR.

[19]  Kurohashi Sadao,et al.  TSUBAKI: An Open Search Engine Infrastructure for Developing Information Access Methodology (特集:情報爆発時代におけるIT基盤技術) , 2011 .

[20]  Yue-Shi Lee,et al.  Combining Multiple Lexical Resources for Chinese Textual Entailment Recognition , 2013, NTCIR.

[21]  Ido Dagan,et al.  The Third PASCAL Recognizing Textual Entailment Challenge , 2007, ACL-PASCAL@ACL.

[22]  Lun-Wei Ku,et al.  Extracting Features for Machine Learning in NTCIR-10 RITE Task , 2013, NTCIR.

[23]  Yusuke Miyao,et al.  BnO at NTCIR-10 RITE: A Strong Shallow Approach and an Inference-based Textual Entailment Recognition System , 2013, NTCIR.

[24]  Sivaji Bandyopadhyay,et al.  Binary-class and Multi-class based Textual Entailment System , 2013, NTCIR.

[25]  Shih-Hung Wu,et al.  CYUT Chinese Textual Entailment Recognition System for NTCIR-10 RITE-2 , 2013, NTCIR.

[26]  Matteo Negri,et al.  Creating a Bi-lingual Entailment Corpus through Translations with Mechanical Turk: $100 for a 10-day Rush , 2010, Mturk@HLT-NAACL.

[27]  Kazutaka Shimada,et al.  KitAi: Textual Entailment Recognition System for NTCIR-10 RITE2 , 2013, NTCIR.

[28]  Ido Dagan,et al.  The Third PASCAL Recognizing Textual Entailment Challenge , 2007, ACL-PASCAL@ACL.

[29]  Marcello Federico,et al.  Using Bilingual Parallel Corpora for Cross-Lingual Textual Entailment , 2011, ACL.

[30]  Hiroshi Kanayama,et al.  IBM Team at NTCIR-10 RITE2: Textual Entailment Using Temporal Dimension Reduction , 2013, NTCIR.

[31]  Hai Zhao,et al.  BCMI-NLP Labeled-Alignment-Based Entailment System for NTCIR-10 RITE-2 Task , 2013, NTCIR.

[32]  Daisuke Satoh,et al.  Expanded Dependency Structure based Textual Entailment Recognition System of NTTDATA for NTCIR10-RITE2 , 2013, NTCIR.

[33]  Satoshi Sato,et al.  Team SKL's Strategy and Experience in RITE2 , 2013, NTCIR.

[34]  Tsuyoshi Okita.  Local Graph Matching with Active Learning for Recognizing Inference in Text at NTCIR-10 , 2013, NTCIR.

[35]  Marcello Federico,et al.  Towards Cross-Lingual Textual Entailment , 2010, NAACL.