论文信息 - Overview of the NTCIR-11 Recognizing Inference in TExt and Validation (RITE-VAL) Task

Overview of the NTCIR-11 Recognizing Inference in TExt and Validation (RITE-VAL) Task

System Valida5on Subtask Language Task Description Acronym Test Data Size Submissions Top Macro-F1 Top Accuracy Top Team (Run Num.) Fact Validation Simplified Chinese Multi-Classification CS-FV 613 12 38.93 44.05 III&CYUT (05) Traditional Chinese Multi-Classification CT-FV 613 15 39.51 44.70 III&CYUT (02) Japanese Binary Classification JA-FV 514 30 61.93 63.23 NUL (03) Passage Search -514 3 ---English Binary Classification EN-FV 188 9 53.17 55.85 BnO (01) System Validation Simplified Chinese Binary Classification CS-SVBC 1,200 23 61.51 62.33 BUPTTeam (05) Multi-Classification CS-SVMC 1,200 18 44.39 51.83 WUST (01) Traditional Chinese Binary Classification CT-SVBC 1,200 17 56.24 56.25 III&CYUT (04) Multi-Classification CT-SVMC 1,200 17 40.54 43.33 III&CYUT (05) Japanese Binary Classification JA-SV 1,379 26 69.59 77.81 NUL (04) Total 170 Active participating team: 23 teams • 11 from Japan • 7 from Taiwan • 4 from China • 1 from Norway • 1 from Vietnam (One team consists of people from Japan and Vietnam.)

[1] Marcello Federico,et al. Towards Cross-Lingual Textual Entailment , 2010, NAACL.

[2] Kalina Bontcheva,et al. Human Language Technologies , 2009, Semantic Knowledge Management.

[3] Marcello Federico,et al. Using Bilingual Parallel Corpora for Cross-Lingual Textual Entailment , 2011, ACL.

[4] Matteo Negri,et al. Creating a Bi-lingual Entailment Corpus through Translations with Mechanical Turk: $100 for a 10-day Rush , 2010, Mturk@HLT-NAACL.

[5] Ido Dagan,et al. The Third PASCAL Recognizing Textual Entailment Challenge , 2007, ACL-PASCAL@ACL.

[6] Noriko Kando,et al. Overview of the Recognizing Inference in Text (RITE-2) at NTCIR-10 , 2013, NTCIR.

[7] Kurohashi Sadao,et al. TSUBAKI: An Open Search Engine Infrastructure for Developing Information Access Methodology (特集:情報爆発時代におけるIT基盤技術) , 2011 .

[8] Doina Tatar,et al. Summarization by Logic Segmentation and Text Entailment , 2008 .

[9] Sanda M. Harabagiu,et al. Methods for Using Textual Entailment in Open-Domain Question Answering , 2006, ACL.

[10] Proceedings of the 11th NTCIR Conference on Evaluation of Information Access Technologies, NTCIR-11, National Center of Sciences, Tokyo, Japan, December 9-12, 2014 , 2014, NTCIR.