Are open-domain question answering technologies useful for information access dialogues?---an empirical study and a proposal of a novel challenge

There are strong expectations for the use of question answering technologies in information access dialogues, such as for information gathering and browsing. In this paper, we empirically examine what kinds of abilities are needed for question answering systems in such situations, and propose a challenge for evaluating those abilities objectively and quantitatively. We also show that existing technologies have the potential to address this challenge. From the empirical study, we found that questions that have values and names as answers account for a majority in realistic information-gathering situations and that those sequences of questions contain a wide range of reference expressions and are sometimes complicated by the inclusion of subdialogues and focus shifts. The challenge proposed is not only novel as an evaluation of the handling of information access dialogues, but also includes several valuable ideas such as categorization and characterization of information access dialogues, and introduces three measures to evaluate various aspects in addressing list-type questions and reference test sets for evaluating context-processing ability in isolation.

[1]  Tsuneaki Kato,et al.  Question Answering Challenge for Information Access Dialogue - Overview of NTCIR4 QAC2 Subtask , 2004, NTCIR.

[2]  Tsuneaki Kato,et al.  Working notes of the fourth NTCIR workshop meeting , 2004 .

[3]  Ellen M. Voorhees,et al.  Overview of the TREC 2002 Question Answering Track , 2003, TREC.

[4]  Tomek Strzalkowski,et al.  HITIQA: A Data Driven Approach to Interactive Question Answering: A Preliminary Report , 2003, New Directions in Question Answering.

[5]  Tsuneaki Kato,et al.  Question Answering Challenge (QAC-1): An Evaluation of Question Answering Tasks at the NTCIR Workshop 3 , 2003, New Directions in Question Answering.

[6]  Katunobu Itou,et al.  Question Answering Using "Common Sense" and Utility Maximization Principle , 2004, NTCIR.

[7]  Junichi Fukumoto,et al.  Rits-QA: List answer detection and Context task with ellipses handling , 2004 .

[8]  Ashwin Ram,et al.  A Theory of Questions and Question Asking , 1991 .

[9]  Sanda M. Harabagiu,et al.  Experiments with Interactive Question Answering in Complex Scenarios , 2004, Workshop On Pragmatics Of Question Answering.

[10]  Ellen M. Voorhees,et al.  The TREC-8 Question Answering Track , 2001, LREC.

[11]  N. H. Beebe.  A Complete Bibliography of ACM Transactions on Asian Language Information Processing , 2007 .

[12]  Yoshio Eriguchi,et al.  NTT DATA Question-Answering Experiment at the NTCIR-3 QAC , 2002, NTCIR.

[13]  Joyce Chai,et al.  Discourse Structure for Context Question Answering , 2004, HLT-NAACL 2004.

[14]  Sanda M. Harabagiu,et al.  Answering Complex, List and Context Questions with LCC's Question-Answering Server , 2001, TREC.

[15]  Ellen M. Voorhees,et al.  Building a question answering test collection , 2000, SIGIR '00.

[16]  Tomek Strzalkowski,et al.  HITIQA: An Interactive Question Answering System: A Preliminary Report , 2003, ACL 2003.