Overview of the NTCIR-10 1CLICK-2 Task

This is an overview of the NTCIR-10 1CLICK-2 task (the second One Click Access task). Given a search query, 1CLICK aims to satisfy the user with a single textual output instead of a ranked list of URLs. Systems are expected to present important pieces of information first and to minimize the amount of text the user has to read. We designed English and Japanese 1CLICK tasks, in which 10 research teams (including two organizers’ teams) participated and submitted 59 runs for Main tasks and a Query Classification subtask. We describe the tasks, test collection, and evaluation methods, and then report official results for NTCIR-10 1CLICK-2.

[1]  Javed A. Aslam,et al.  A unified model for metasearch and the efficient evaluation of retrieval systems via the hedge algorithm , 2003, SIGIR '03.

[2]  Tetsuya Sakai,et al.  Exploring semi-automatic nugget extraction for Japanese one click access evaluation , 2013, SIGIR.

[3]  Hiroya Takamura,et al.  TTOKU Summarization Based Systems at NTCIR-10 1CLICK-2 task , 2013, NTCIR.

[4]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[5]  Young-In Song,et al.  Click the search button and be happy: evaluating direct and immediate information access , 2011, CIKM '11.

[6]  Noriko Kando,et al.  Overview of the NTCIR-7 ACLIA Tasks: Advanced Cross-Lingual Information Access , 2008, NTCIR.

[7]  Meng Zhao,et al.  Information Extraction based Approach for the NTCIR-9 1CLICK Task , 2011, NTCIR.

[8]  Javed A. Aslam,et al.  IR system evaluation using nugget-based test collections , 2012, WSDM '12.

[9]  Jane Li,et al.  Good abandonment in mobile and PC internet search , 2009, SIGIR.

[10]  Jimmy J. Lin,et al.  Different Structures for Evaluating Answers to Complex Questions: Pyramids Won’t Topple, and Neither Will Human Assessors , 2007, ACL.

[11]  Young-In Song,et al.  Overview of NTCIR-9 1CLICK , 2011, NTCIR.

[12]  Tetsuya Sakai,et al.  TTOKU Summarization Based Systems at NTCIR-9 1CLICK task , 2011, NTCIR.

[13]  Ben Carterette,et al.  Multiple testing in statistical analysis of systems-based information retrieval experiments , 2012, TOIS.

[14]  Andrei Z. Broder,et al.  Identifying and Filtering Near-Duplicate Documents , 2000, CPM.

[15]  H. Sebastian Seung,et al.  Query by committee , 1992, COLT '92.

[16]  Noriko Kando,et al.  On information retrieval metrics designed for evaluation with incomplete relevance assessments , 2008, Information Retrieval.

[17]  Charles L. A. Clarke,et al.  Efficient construction of large test collections , 1998, SIGIR '98.

[18]  Conclusions , 1989 .

[19]  Young-In Song,et al.  MSRA at NTCIR-10 1CLICK-2 , 2013, NTCIR.

[20]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[21]  Ran El-Yaniv,et al.  Online Choice of Active Learning Algorithms , 2003, J. Mach. Learn. Res..

[22]  Jing He,et al.  Hunter Gatherer: UdeM at 1CLICK-2 , 2013, NTCIR.

[23]  Geoffrey Zweig,et al.  Syntactic Clustering of the Web , 1997, Comput. Networks.

[24]  Masaharu Yoshioka.  Query Classification by Using Named Entity Recognition Systems and Clue Keywords , 2013, NTCIR.

[25]  Javed A. Aslam.  Improving Algorithms for Boosting , 2000, COLT.

[26]  James Allan,et al.  Minimal test collections for retrieval evaluation , 2006, SIGIR.

[27]  Takashi Yukawa,et al.  Query Classification System Based on Snippet Summary Similarities for NTCIR-10 1CLICK-2 Task , 2013, NTCIR.

[28]  Tetsuya Sakai,et al.  One Click One Revisited: Enhancing Evaluation Based on Information Units , 2012, AIRS.

[29]  Javed A. Aslam,et al.  Constructing test collections by inferring document relevance via extracted relevant information , 2012, CIKM.

[30]  Hirokazu Kato,et al.  XML Element Retrieval@1CLICK-2 , 2013, NTCIR.