论文信息 - The Expert of Oz: A Two-sided Study Paradigm for Intelligent Systems

The Expert of Oz: A Two-sided Study Paradigm for Intelligent Systems

We present the paradigm of Expert of Oz studies for the formative evaluation of hypothetical AI systems. These studies follow the principle of Wizard of Oz studies but use a human expert for simulating the AI. This allows the experimenter to not only investigate the user's behavior with and reaction to such a system but also to observe and analyze the performance of the expert in the situation at hand. As a consequence, we can learn both about interaction with the hypothetical AI system and about the required AI functionality at the same time and in realistic interaction scenarios.

Andreas Butz | Malin Eiband | Annika Kaltenhauser

[1] Andreas Butz,et al. "You Have to Piece the Puzzle Together": Implications for Designing Decision Support in Intensive Care , 2020, Conference on Designing Interactive Systems.

[2] Mark Bilandzic,et al. Bringing Transparency Design into Practice , 2018, IUI.

[3] Lennart Molin,et al. Wizard-of-Oz prototyping for co-operative interaction design of graphical user interfaces , 2004, NordiCHI '04.

[4] Emilee J. Rader,et al. Explanations as Mechanisms for Supporting Algorithmic Transparency , 2018, CHI.

[5] Saul Greenberg,et al. Prototyping an intelligent agent through Wizard of Oz , 1993, INTERCHI.

[6] Tim Miller,et al. Explanation in Artificial Intelligence: Insights from the Social Sciences , 2017, Artif. Intell..

[7] J. F. Kelley,et al. An iterative design methodology for user-friendly natural language office information applications , 1984, TOIS.