WoZ Simulation of Interactive Question Answering

QACIAD (Question Answering Challenge for Information Access Dialogue) is an evaluation framework for measuring interactive question answering (QA) technologies. It assumes that users interactively collect information using a QA system for writing a report on a given topic and evaluates, among other things, the capabilities needed under such circumstances. This paper reports an experiment for examining the assumptions made by QACIAD. In this experiment, dialogues under the situation that QACIAD assumes are collected using WoZ (Wizard of Oz) simulating, which is frequently used for collecting dialogue data for designing speech dialogue systems, and then analyzed. The results indicate that the setting of QACIAD is real and appropriate and that one of the important capabilities for future interactive QA systems is providing cooperative and helpful responses.

[1]  Michael Brady,et al.  Cooperative Responses From a Portable Natural Language Database Query System , 1983 .

[2]  Noriko Kando,et al.  Are open-domain question answering technologies useful for information access dialogues?---an empirical study and a proposal of a novel challenge , 2005, TALIP.

[3]  Ellen M. Voorhees,et al.  Overview of the TREC 2004 Novelty Track. , 2005 .

[4]  Tomek Strzalkowski,et al.  HITIQA: A Data Driven Approach to Interactive Question Answering: A Preliminary Report , 2003, New Directions in Question Answering.

[5]  Tsuneaki Kato,et al.  An Overview of NTCIR-5 QAC3 , 2005, NTCIR.

[6]  Joyce Chai,et al.  Discourse Structure for Context Question Answering , 2004, HLT-NAACL 2004.

[7]  Ellen M. Voorhees,et al.  Building a question answering test collection , 2000, SIGIR '00.

[8]  Samuel Jerrold Kaplan,et al.  Cooperative responses from a portable natural language data base query system. , 1979 .

[9]  Tsuneaki Kato,et al.  Question Answering Challenge for Information Access Dialogue - Overview of NTCIR4 QAC2 Subtask , 2004, NTCIR.

[10]  Sanda M. Harabagiu,et al.  Experiments with Interactive Question Answering in Complex Scenarios , 2004, Workshop On Pragmatics Of Question Answering.

[11]  Nigel Gilbert,et al.  Simulating speech systems , 1991 .

[12]  Tsuneaki Kato,et al.  Handling Information Access Dialogue through QA Technologies - A novel challenge for open-domain question answering , 2004 .

[13]  Philip R. Cohen,et al.  Plans as Complex Mental Attitudes , 2003 .