Wizard-of-Oz Data Collection for Perception and Interaction in Multi-User Environments

In this paper we present the setup of an extensive Wizard-of-Oz environment used for the data collection and the development of a dialogue system. The envisioned Perception and Interaction Assistant will act as an independent dialogue partner. Passively observing the dialogue between the two human users with respect to a limited domain, the system should take the initiative and get meaningfully involved in the communication process when required by the conversational situation. The data collection described here involves audio and video data. We aim at building a rich multi-media data corpus to be used as a basis for our research which includes, inter alia, speech and gaze direction recognition, dialogue modelling and proactivity of the system. We further aspire to obtain data with emotional content to perfom research on emotion recognition, psychopysiological and usability analysis.

[1]  Norbert Krüger,et al.  Determination of face position and pose with a learned representation based on labelled graphs , 1997, Image Vis. Comput..

[2]  George N. Votsis,et al.  Emotion recognition in human-computer interaction , 2001, IEEE Signal Process. Mag..

[3]  P Sinha,et al.  Last but Not Least , 2000, Perception.

[4]  Roberto Cipolla,et al.  Determining the gaze of faces in images , 1994, Image Vis. Comput..

[5]  Wolfgang Wahlster,et al.  Smartkom: multimodal communication with a life- like character , 2001, INTERSPEECH.

[6]  L. Jacobson Here's Looking At You, Kid. , 2000 .

[7]  Wolfgang Wahlster,et al.  Verbmobil: Foundations of Speech-to-Speech Translation , 2000, Artificial Intelligence.

[8]  Alexander H. Waibel,et al.  The connector: facilitating context-aware communication , 2005, ICMI '05.

[9]  W. Keller,et al.  Last but Not Least Regulated Poly(A) Tail Formation , 1999, Cell.

[10]  Wang Ke,et al.  Face pose estimation with a knowledge-based model , 2003, International Conference on Neural Networks and Signal Processing, 2003. Proceedings of the 2003.

[11]  Alexander H. Waibel CHIL - Computers in the Human Interaction Loop , 2005, MVA.

[12]  V. Bruce,et al.  Do the eyes have it? Cues to the direction of social attention , 2000, Trends in Cognitive Sciences.